[Dec-2023] Latest Databricks-Certified-Professional-Data-Engineer Exam Dumps for Pass Guaranteed
Reliable Databricks Certification Databricks-Certified-Professional-Data-Engineer Dumps PDF Dec 18, 2023 Recently Updated Questions
NEW QUESTION # 35
Delete records from the transactions Delta table where transactionDate is greater than current timestamp?
- A. DELETE FROM transactions if transctionDate > current_timestamp()
- B. DELET FROM transactions where transactionDate GE current_timestamp()
- C. DELETE FROM transactions where transactionDate > current_timestamp() KEEP_HISTORY
- D. DELETE FROM transactions where transactionDate > current_timestamp()
- E. DELETE FROM transactions FORMAT DELTA where transactionDate > cur-renct_timestmap()
Answer: D
NEW QUESTION # 36
A data engineering team needs to query a Delta table to extract rows that all meet the same condi-tion.
However, the team has noticed that the query is running slowly. The team has already tuned the size of the
data files. Upon investigating, the team has concluded that the rows meeting the condition are sparsely located
throughout each of the data files.
Based on the scenario, which of the following optimization techniques could speed up the query?
- A. Bin-packing
- B. Z-Ordering
- C. Tuning the file size
- D. Write as a Parquet file
- E. Data skipping
Answer: B
NEW QUESTION # 37
An engineering manager uses a Databricks SQL query to monitor their team's progress on fixes related to
customer-reported bugs. The manager checks the results of the query every day, but they are manually
rerunning the query each day and waiting for the results.
Which of the following approaches can the manager use to ensure the results of the query are up-dated each
day?
- A. They can schedule the query to refresh every 1 day from the query's page in Databricks SQL
- B. They can schedule the query to refresh every 12 hours from the SQL endpoint's page in Databricks SQL
- C. They can schedule the query to run every 12 hours from the Jobs UI
- D. They can schedule the query to refresh every 1 day from the SQL endpoint's page in Databricks SQL
- E. They can schedule the query to run every 1 day from the Jobs UI
Answer: A
NEW QUESTION # 38
You have noticed the Data scientist team is using the notebook versioning feature with git integra-tion, you have recommended them to switch to using Databricks Repos, which of the below reasons could be the reason the why the team needs to switch to Databricks Repos.
- A. Databricks Repos allows multiple users to make changes
- B. Databricks Repos allows merge and conflict resolution
- C. Databricks Repos allow you to add comments and select the changes you want to commit.
- D. Databricks Repos automatically saves changes
- E. Databricks Repos has a built-in version control system
Answer: C
Explanation:
Explanation
The answer is Databricks Repos allow you to add comments and select the changes you want to commit.
NEW QUESTION # 39
When you drop a managed table using SQL syntax DROP TABLE table_name how does it impact metadata, history, and data stored in the table?
- A. Drops table from meta store and data from storage but keeps metadata and history in storage
- B. Drops table from meta store, drops metadata, history, and data in storage.
- C. Drops table and history but keeps meta data and data in storage
- D. Drops table from meta store, meta data and history but keeps the data in storage
- E. Drops table but keeps meta data, history and data in storage
Answer: B
Explanation:
Explanation
For a managed table, a drop command will drop everything from metastore and storage.
See the below image to understand the differences between dropping an external table.
Diagram Description automatically generated
NEW QUESTION # 40
A Spark job is taking longer than expected. Using the Spark UI, a data engineer notes that the Min, Median, and Max Durations for tasks in a particular stage show the minimum and median time to complete a task as roughly the same, but the max duration for a task to be roughly 100 times as long as the minimum.
Which situation is causing increased duration of the overall job?
- A. Task queueing resulting from improper thread pool assignment.
- B. Skew caused by more data being assigned to a subset of spark-partitions.
- C. Spill resulting from attached volume storage being too small.
- D. Network latency due to some cluster nodes being in different regions from the source data
- E. Credential validation errors while pulling data from an external system.
Answer: B
Explanation:
Explanation
This is the correct answer because skew is a common situation that causes increased duration of the overall job. Skew occurs when some partitions have more data than others, resulting in uneven distribution of work among tasks and executors. Skew can be caused by various factors, such as skewed data distribution, improper partitioning strategy, or join operations with skewed keys. Skew can lead to performance issues such as long-running tasks, wasted resources, or even task failures due to memory or disk spills. Verified References:
[Databricks Certified Data Engineer Professional], under "Performance Tuning" section; Databricks Documentation, under "Skew" section.
NEW QUESTION # 41
Which of the following developer operations in the CI/CD can only be implemented through a GIT provider when using Databricks Repos.
- A. Create and edit code
- B. Trigger Databricks Repos pull API to update the latest version
- C. Pull request and review process
- D. Commit and push code
- E. Create a new branch
Answer: C
Explanation:
Explanation
The answer is Pull request and review process, please note: the question is asking for steps that are being implemented in GIT provider not Databricks Repos.
See below diagram to understand the role of Databricks Repos and Git provider plays when building a CI/CD workdlow.
All the steps highlighted in yellow can be done Databricks Repo, all the steps highlighted in Gray are done in a git provider like Github or Azure Devops.
Diagram Description automatically generated
Bottom of Form
Top of Form
NEW QUESTION # 42
A Delta Lake table was created with the below query:
Realizing that the original query had a typographical error, the below code was executed:
ALTER TABLE prod.sales_by_stor RENAME TO prod.sales_by_store
Which result will occur after running the second command?
- A. A new Delta transaction log Is created for the renamed table.
- B. The table reference in the metastore is updated and no data is changed.
- C. All related files and metadata are dropped and recreated in a single ACID transaction.
- D. The table name change is recorded in the Delta transaction log.
- E. The table reference in the metastore is updated and all data files are moved.
Answer: B
Explanation:
Explanation
The query uses the CREATE TABLE USING DELTA syntax to create a Delta Lake table from an existing Parquet file stored in DBFS. The query also uses the LOCATION keyword to specify the path to the Parquet file as /mnt/finance_eda_bucket/tx_sales.parquet. By using the LOCATION keyword, the query creates an external table, which is a table that is stored outside of the default warehouse directory and whose metadata is not managed by Databricks. An external table can be created from an existing directory in a cloud storage system, such as DBFS or S3, that contains data files in a supported format, such as Parquet or CSV.
The result that will occur after running the second command is that the table reference in the metastore is updated and no data is changed. The metastore is a service that stores metadata about tables, such as their schema, location, properties, and partitions. The metastore allows users to access tables using SQL commands or Spark APIs without knowing their physical location or format. When renaming an external table using the ALTER TABLE RENAME TO command, only the table reference in the metastore is updated with the new name; no data files or directories are moved or changed in the storage system. The table will still point to the same location and use the same format as before. However, if renaming a managed table, which is a table whose metadata and data are both managed by Databricks, both the table reference in the metastore and the data files in the default warehouse directory are moved and renamed accordingly. Verified References:
[Databricks Certified Data Engineer Professional], under "Delta Lake" section; Databricks Documentation, under "ALTERTABLE RENAME TO" section; Databricks Documentation, under "Metastore" section; Databricks Documentation, under "Managed and external tables" section.
NEW QUESTION # 43
Which of the following SQL commands are used to append rows to an existing delta table?
- A. UPDATE table_name
- B. APPEND INTO table_name
- C. COPY DELTA INTO table_name
- D. INSERT INTO table_name
- E. APPEND INTO DELTA table_name
Answer: D
Explanation:
Explanation
The answer is INSERT INTO table_name
Insert adds rows to an existing table, this is very similar to add rows a traditional Database or Da-tawarehouse.
NEW QUESTION # 44
Which of the following is true, when building a Databricks SQL dashboard?
- A. A dashboard can only use results from one query
- B. A dashboard can only connect to one schema/Database
- C. Only one visualization can be developed with one query result
- D. A dashboard can only have one refresh schedule
- E. More than one visualization can be developed using a single query result
Answer: E
Explanation:
Explanation
the answer is, More than one visualization can be developed using a single query result.
In the query editor pane + Add visualization tab can be used for many visualizations for a single query result.
Graphical user interface, text, application Description automatically generated
NEW QUESTION # 45
When investigating a data issue you realized that a process accidentally updated the table, you want to query the same table with yesterday's version of the data so you can review what the prior version looks like, what is the best way to query historical data so you can do your analysis?
- A. TIME_TRAVEL FROM table_name WHERE time_stamp = date_sub(current_date(), 1)
- B. SHOW HISTORY table_name AS OF date_sub(current_date(), 1)
- C. SELECT * FROM table_name TIMESTAMP AS OF date_sub(current_date(), 1)
- D. DISCRIBE HISTORY table_name AS OF date_sub(current_date(), 1)
- E. SELECT * FROM TIME_TRAVEL(table_name) WHERE time_stamp = 'timestamp'
Answer: C
Explanation:
Explanation
The answer is SELECT * FROM table_name TIMESTAMP as of date_sub(current_date(), 1) FYI, Time travel supports two ways one is using timestamp and the second way is using version number, Timestamp:
1.SELECT count(*) FROM my_table TIMESTAMP AS OF "2019-01-01"
2.SELECT count(*) FROM my_table TIMESTAMP AS OF date_sub(current_date(), 1)
3.SELECT count(*) FROM my_table TIMESTAMP AS OF "2019-01-01 01:30:00.000" Version Number:
1.SELECT count(*) FROM my_table VERSION AS OF 5238
2.SELECT count(*) FROM my_table@v5238
3.SELECT count(*) FROM delta.`/path/to/my/table@v5238`
https://databricks.com/blog/2019/02/04/introducing-delta-time-travel-for-large-scale-data-lakes.html
NEW QUESTION # 46
You are currently asked to work on building a data pipeline, you have noticed that you are currently working on a very large scale ETL many data dependencies, which of the following tools can be used to address this problem?
- A. DELTA LIVE TABLES
- B. STRUCTURED STREAMING with MULTI HOP
- C. SQL Endpoints
- D. JOBS and TASKS
- E. AUTO LOADER
Answer: A
Explanation:
Explanation
The answer is, DELTA LIVE TABLES
DLT simplifies data dependencies by building DAG-based joins between live tables. Here is a view of how the dag looks with data dependencies without additional meta data,
1.create or replace live view customers
2.select * from customers;
3.
4.create or replace live view sales_orders_raw
5.select * from sales_orders;
6.
7.create or replace live view sales_orders_cleaned
8.as
9.select sales.* from
10.live.sales_orders_raw s
11. join live.customers c
12.on c.customer_id = s.customer_id
13.where c.city = 'LA';
14.
15.create or replace live table sales_orders_in_la
16.selects from sales_orders_cleaned;
Above code creates below dag
Documentation on DELTA LIVE TABLES,
https://databricks.com/product/delta-live-tables
https://databricks.com/blog/2022/04/05/announcing-generally-availability-of-databricks-delta-live-tables-dlt.htm DELTA LIVE TABLES, addresses below challenges when building ETL processes
1.Complexities of large scale ETL
a.Hard to build and maintain dependencies
b.Difficult to switch between batch and stream
2.Data quality and governance
a.Difficult to monitor and enforce data quality
b.Impossible to trace data lineage
3.Difficult pipeline operations
a.Poor observability at granular data level
b.Error handling and recovery is laborious
NEW QUESTION # 47
When scheduling Structured Streaming jobs for production, which configuration automatically recovers from query failures and keeps costs low?
- A. Cluster: New Job Cluster;
Retries: None;
Maximum Concurrent Runs: 1 - B. Cluster: Existing All-Purpose Cluster;
Retries: None;
Maximum Concurrent Runs: 1 - C. Cluster: New Job Cluster;
Retries: Unlimited;
Maximum Concurrent Runs: Unlimited - D. Cluster: Existing All-Purpose Cluster;
Retries: Unlimited;
Maximum Concurrent Runs: 1 - E. Cluster: Existing All-Purpose Cluster;
Retries: Unlimited;
Maximum Concurrent Runs: 1
Answer: A
Explanation:
Explanation
This is the best configuration for scheduling Structured Streaming jobs for production, as it automatically recovers from query failures and keeps costs low. A new job cluster is created for each run of the job and terminated when the job completes, which saves costs and avoids resource contention. Retries are not needed for Structured Streaming jobs, as they can automatically recover from failures using checkpointing and write-ahead logs. Maximum concurrent runs should be set to 1 to avoid duplicate output or data loss. Verified References: Databricks Certified Data Engineer Professional, under "Monitoring & Logging" section; Databricks Documentation, under "Schedule streaming jobs" section.
NEW QUESTION # 48
One of the team members Steve who has the ability to create views, created a new view called re-gional_sales_vw on the existing table called sales which is owned by John, and the second team member Kevin who works with regional sales managers wanted to query the data in region-al_sales_vw, so Steve granted the permission to Kevin using command GRANT VIEW, USAGE ON regional_sales_vw to [email protected] but Kevin is still unable to access the view?
- A. Table access control is not enabled on the table and view
- B. Kevin is not the owner of the sales table
- C. Kevin needs owner access on the view regional_sales_vw
- D. Kevin needs select access on the table sales
- E. Steve is not the owner of the sales table
Answer: E
Explanation:
Explanation
Ownership determines whether or not you can grant privileges on derived objects to other users, since Steve is not the owner of the underlying sales table, he can not grant access to the table or data in the table indirectly.
Only owner(user or group) can grant access to a object
https://docs.microsoft.com/en-us/azure/databricks/security/access-control/table-acls/object-privileges#a-user-has Data object privileges - Azure Databricks | Microsoft Doc
NEW QUESTION # 49
Which of the statements are correct about lakehouse?
- A. Lakehouse does not support ACID
- B. In Lakehouse Storage and compute are coupled
- C. Lakehouse supports schema enforcement and evolution
- D. Lakehouse only supports end-to-end streaming workloads and Data warehouses support Batch workloads
- E. Lakehouse only supports Machine learning workloads and Data warehouses support BI workloads
Answer: C
Explanation:
Explanation
The answer is Lakehouse supports schema enforcement and evolution,
Lakehouse using Delta lake can not only enforce a schema on write which is contrary to traditional big data systems that can only enforce a schema on read, it also supports evolving schema over time with the ability to control the evolution.
For example below is the Dataframe writer API and it supports three modes of enforcement and evolution, Default: Only enforcement, no changes are allowed and any schema drift/evolution will result in failure.
Merge: Flexible, supports enforcement and evolution
* New columns are added
* Evolves nested columns
* Supports evolving data types, like Byte to Short to Integer to Bigint How to enable:
* DF.write.format("delta").option("mergeSchema", "true").saveAsTable("table_name")
* or
* spark.databricks.delta.schema.autoMerge = True ## Spark session
Overwrite: No enforcement
* Dropping columns
* Change string to integer
* Rename columns
How to enable:
* DF.write.format("delta").option("overwriteSchema", "True").saveAsTable("table_name") What Is a Lakehouse? - The Databricks Blog Graphical user interface, text, application Description automatically generated
NEW QUESTION # 50
......
Latest 2023 Realistic Verified Databricks-Certified-Professional-Data-Engineer Dumps: https://www.itexamsimulator.com/Databricks-Certified-Professional-Data-Engineer-brain-dumps.html
