How Can We Create a Snowflake Python Worksheet?

Snowflake has recently released the Snowflake Python Worksheets which is in public preview at present. This will make it much more easier than ever to integrate Snowpark Python directly into your Snowflake environment.

Today, we will cover how to set up your Python worksheets, write Python code, run it efficiently, and deploy it as a stored procedure within your Snowflake environment.

Prerequisites

1. Login to Snowflake using an admin account and switch to the ORGADMIN role. This role is required to enable Python packages.
2. Under “Admin” section and click on “Billing & Terms”
3. Under “Anaconda –> Anaconda Python Packages”, click “Enable”
4. Click on “Acknowledge & Continue”, This will enable the Python feature for the Snowflake worksheets

Creating Python Worksheets in Snowflake

STEP 1: Log in to the Snowflake (Snowsight UI) and navigate to the Worksheets located in the left-side panel. Click on “+” button and select the “Python Worksheet”.

STEP 2: Select the Database & Virtual Warehouse for the worksheet.

STEP 3: Now, write some python code inside the Snowflake Python worksheet. In the example below, we are generating the five employee records using a random function. Copy the below sample code into your worksheet and click on Run button. 

How Do We Add Custom Python Packages?

What if you can’t find the package in the Snowflake Anaconda? Or, you have created a custom Python file or package and wanted to import it into your Snowflake worksheet? Stated below are the steps to import custom packages:

Step 1: In the worksheet tab, under the “Packages”, select “Stage Packages” and import the required Python file or package which is stage location.

For example, @YourStage/path/to/package.py

Step 2: Now, select Import to add your package to the list of installed packages. You can now use import statements to use the package in your Python worksheet.

Stream & Merge for Incremental Loading in Snowflake

Is manual data loading slowing down operations in your organization? Discover Snowflake’s approach to data warehousing. By leveraging Stream and Merge objects, it enables effortless incremental loading, reducing time and resources compared to full loads.

How Do We Deploy the Python Worksheet as Snowflake Stored Procedure?

Step 1: On the right-hand corner of the worksheet, click on the “Deploy” and input stored procedure name. Then, click on the Deploy button to create as a Snowflake Stored Procedure.

Step 2: Call the Stored Procedure as below once it is created successfully.

Command: CALL TEST_DB1.PUBLIC.SAMPLE_EMPDATA () 

Step 3: Now, you have successfully deployed your python script as a Snowflake Stored Procedure.

Benefits of Snowflake Python Worksheet

• Develop and test your code without the need to install any dependencies locally.
• Easily set up your worksheet for development by just selecting a DB and schema, configuring roles and privileges, and adding any required packages.
• Easily write Snowpark code in your Python worksheet to perform any kind of data processing tasks, such as extracting data from stages or database objects in Snowflake, transforming the data, and storing the transformed data in Snowflake.
• Convert your code to a stored procedure and build a data pipeline within Snowflake, without having to leave the Snowflake environment.

Limitations:

• Log levels lower than WARN do not appear in the Output for a Snowflake Python worksheet by default, but you can use a logging library, such as the logging module to set the level of messages logged.
• Snowflake Python worksheets do not support breakpoints or running only portions of the Python code in a worksheet.
• Snowflake Python worksheet operates within Snowflake rather than in a local development environment;
• Snowflake Python worksheets cannot display images or web pages generated by Python code.
• Snowpark Python uses Python 3.8, meaning that any package or library that requires a different version of Python may not be compatible with Snowflake Python Worksheets. Refer to the official Snowflake documentation for more information.

Snowflake Python Worksheets: Streamlined Execution & Deployment

Snowflake Python Worksheets are a game-changing feature that was just recently made available to public preview. These worksheets made it easier and faster to execute your Python code in the Snowflake environment. It also comes with features that support Snowpark and other third-party libraries. On top of that, it also supports IntelliSense with auto-complete, making it even easier to use. 

In this article, we covered all the essentials related to Snowflake Python Worksheets. We discussed how to prepare roles and packages, set up your Python worksheet, wrote a Python script, ran it efficiently, and deployed it as a Stored Procedure with just one click!

Streamlining ETL Pipeline with Snowflake, AWS, and PySpark

Are you looking to streamline your data pipeline? Learn to set up an efficient ETL pipeline using Snowflake, AWS, and PySpark with this insightful guide.