Name: 10 Best Practices for Data Engineers
Start: 2020-04-23T17:00:00Z
End: 2020-04-23T17:00:50.000Z
Location: BrightTALK
Rating: 4.2

Snowflake makes enterprise AI easy, efficient and trusted. Thousands of companies around the globe, including hundreds of the world’s largest, use Snowflake’s AI Data Cloud to share data, build applications, and power their business with AI. The era of enterprise AI is here. Learn more at snowflake.com (NYSE: SNOW).

In today’s AI-powered world, organizations must continuously find new ways to accelerate time to value from all their data to stay ahead of the competition. For decades, data and AI platforms have required organizations to spend valuable time and resources on cluster spin-up and spin-down, disruptive upgrades, manual performance tuning and laborious optimizations.

Snowflake revolutionized the data world with our virtual warehouses that had automated cluster management, maintenance, upgrades and regular performance improvements — all without any downtime — for a truly easy-to-use service. Continuously improving our compute to make it even faster and even easier to use, with better price/performance has been a key priority. 

Register for this live demo event to learn more about:
 

- How Snowflake is continuously improving performance for customers through our innovations
- The latest warehouse and container services enhancements in Snowflake
- How customers are leveraging the latest compute innovations to drive business value and accelerate innovation
- See new innovations in action with 3 live demos

Snowflake Connect: Platform - High Performance Compute for the AI Era

To effectively govern your AI models end to end, you need visibility and controls across the entire data lifecycle. This includes ensuring data quality for model inputs, using data lineage to track dependencies and workflows, and enforcing safeguards on model outputs. 

Join this session to learn best practices for governing models with Snowflake to:

- Improve your data integrity
- Increase transparency into your AI pipeline
- Ensure responsible AI deployment

Best Practices for AI Governance and Security in Snowflake

Why attend: Data is often silo’d between various teams across an organization, spread across engines, clouds, regions and modalities. In this data mesh virtual event, we will show you how to break down silos by securely sharing data products by leveraging a data mesh architecture.  
  

Discover how to:    

- Build a unified data mesh by securely connecting and governing data across clouds and regions  

- Leverage open table formats like Apache Iceberg to enable bidirectional data sharing, without sacrificing governance  

- Power AI and machine learning initiatives by using secure data sharing, Snowpark API, Snowflake Cortex and Horizon Catalog.  


Who should attend:   

Data architects, data engineers and data leaders building collaborative strategies for more effective data sharing, faster time-to-market and cost efficiency.

Unlocking AI with an Interoperable Data Mesh: Democratize Data Products

Data is powerful, but only when you can transform it into actionable insights. With Snowflake and Microsoft Power BI, you can accelerate analytics for all users and queries, create interactive reports and dashboards to deliver real-time insights for informed decision-making.

In this step-by-step session, we’ll show you how to efficiently set up Snowflake with Power BI to visualize, analyze and report on your most important business data – all without any need for data movement. 

You’ll learn how to:

- Understand various methods of connecting Power BI with Snowflake.
- Understand the fundamentals and benefits of a star schema design.
- Connect Snowflake and Power BI in seconds.
- Easily profile data with Snowsight workspaces.
- Transform raw data into an optimal format for analysis within Power BI. 
- Analyze Snowflake data from Power BI and talk to your data with Power BI Copilot.
- Use the new Iceberg Tables with Power BI to unlock insights.
- Best practices and performance improvements.

Virtual Hands-on Lab: End-to-End Analytics with Snowflake and Power BI

Snowflake’s support for open formats and integrated data engineering capabilities empower data teams with a new paradigm for architecting more efficient data platforms. This new paradigm aims to overcome data fragmentation without compromising openness and flexibility, providing agent-ready bi-directional data connections, and empowering every data engineer — from novice to expert — to code and automate pipelines with greater confidence. The result: a connected business ecosystem that surfaces competitive opportunities faster, diagnoses challenges earlier, and democratized governed data products and AI.

Join this virtual event to learn how you can:

- Build a new architecture paradigm for AI
- Unlock in-the-moment insights with AI
- Use emerging open architecture patterns for greater efficiency and AI
- Power the new architecture pattern

Data Engineering Connect: Architect for AI using Open Formats

You can now run Apache Spark DataFrame code on Snowflake. Join our deep dive into Snowpark Connect for Apache Spark, a new public preview.

Built on Spark Connect, this feature allows Apache Spark clients like PySpark to seamlessly connect to the Snowflake platform. You can execute all your modern Apache Spark DataFrame, Spark SQL, and user-defined function (UDF) code directly against your data (in Snowflake, Iceberg Tables, or external storage) using Snowpark execution. Say goodbye to the complexities of maintaining separate Apache Spark clusters—no more managing dependencies, wrestling with version compatibility, or planning upgrades. 

In this webinar, we’ll go deep into the feature, how it works, and how to get started. What you’ll learn:

- The architecture: Understand how Snowpark Connect leverages Spark Connect to bridge your Apache Spark applications with the Snowflake engine.
- Developer experience: Watch live demos of Apache Spark code—including DataFrames, SQL, and UDFs—running directly on Snowflake’s managed compute.
- Check compatibility: Learn about the compatibility tooling in the SMA to find the best workloads and how to connect your existing Apache Spark DataFrame applications to Snowflake.

Demo: Snowpark Connect for Apache Spark Deep Dive

Data engineers are often working with complex integration needs but are constrained by rigid pipelines and resources. This largely prohibited the speed of innovation, especially to deliver new AI initiatives. Now with Snowflake Openflow, data engineers can build open, extensible and secure data integrations that enable real-time, scalable and bi-directional data movement. Powered by Apache NiFi, Openflow offers customers the option to use self-contained deployments and runtimes in Snowpark Container Services, as well as the ability to run bring your own cloud (BYOC) deployments in the customer’s cloud VPC. In this technical session, we will dive into the platform architecture.

Learn how to bring all your data together with Snowflake Openflow to build with limitless interoperability and deployment flexibility, no matter if it’s structured, unstructured, batch or streaming data. Make your integration pipelines ready for AI and the future.

Effortless Data Integration, Ready for AI: A Technical Deep Dive of Snowflake Openflow

Get a practical introduction to building end-to-end data pipelines in Python. This hour-long webinar will give you hands-on experience using Snowflake for scalable data processing. Learn how to connect to data, run transformations, orchestrate workflows, and automate pipeline execution with Snowflake.

In this session, we’ll show you how to: 

- Ingest csv files from AWS S3 bucket into Snowflake tables
- Load the tables into a Snowpark DataFrame
- Join tables and run aggregations within Snowpark
- Explore data with visualizations with Snowflake Notebooks
- Use pandas on Snowflake to read files from a git repository, run calculations, and visualize results

Virtual hands-on lab: Build End-to-End Data Pipelines in Python with Snowflake

As AI technology improves and infrastructure investments skyrocket, the urgency for enterprise leaders to move from experimentation to production and ROI also has grown. Agentic AI, in particular, has captured the market’s attention by offering a powerful path to unlock meaningful business value by automating tasks, generating insights and improving customer experience. 

Join leaders from Accenture, Amazon Web Services (AWS), Luminate and Snowflake for a lively discussion on the future of agentic AI, practical tips to build AI agents and bring them into production securely at scale, and real-world examples of agentic deployments in the enterprise.

The Path to Production-Ready AI Agents

Open lakehouse architecture is revolutionizing data management, offering unmatched scalability, flexibility, and interoperability for modern analytics. 

Komodo Health’s AI-powered platform transforms real-world healthcare data into actionable insights for life sciences, payers, and providers. Processing 2M+ daily queries across 1,000+ environments, Komodo enables seamless decision-making for 330M+ patients. Join this session to explore how they leveraged Snowflake-managed Iceberg tables and an open lakehouse architecture to enhance interoperability and power their suite of applications. 

In this session, we will discuss: 

- Considerations for architecting an open lakehouse 
- How to transform data into Iceberg tables for analytics 
- How to securely share Iceberg tables via secure data sharing 
- How to leverage Snowflake Open Catalog to manage data 
- Implementation tips and learnings

Your Data, Your Way: Enabling Analytics with an Open Lakehouse

This hands-on lab explores several Time Series features using FactSet Tick Data from Snowflake Marketplace, including TIME_SLICE, ASOF_JOIN and RANGE BETWEEN for insights into trade data. Learn how to build an AI-powered chatbot to help drive additional insights from the data. 

What you will learn: 
--Leveraging powerful SQL functions such as TIME_SLICE, ASOF JOIN, RANGE BETWEEN and more 
--Analyzing Time Series data using Snowflake Notebooks 
--Accessing data from Snowflake Marketplace

Virtual Hands-on Lab: Time Series Analytics and LLM powered Chatbot with Pricing Data

Want to migrate to Snowflake in less than 90 minutes? Join our hands-on lab. 

Here's what you'll accomplish: 
--Leverage SnowConvert and the Snowpark Migration Accelerator to assess, move, and validate your data warehouse SQL and Spark pipelines to Snowflake. 
--Migrate data to Snowflake from various sources, including other data warehouses, data lakes, and on-premises systems. 
--Explore the possibility of running your existing Jupyter notebooks utilizing Spark directly within Snowflake.

Virtual Hands-on Lab: End-to-End Migration to Snowflake in Action: Data and Pipelines

Many data teams are overloaded with requests, struggling to deliver the right data to the right teams fast enough. This leads to organizational bottlenecks, slower time to market and limited scalability.  

To overcome these challenges, enterprises are adopting data mesh, a decentralized approach that aims to make data products easily discoverable and accessible, with a high level of governance, reducing the pressure on central teams.  

By improving data sharing across business units in a self-service manner, companies can increase data agility across their organization to drive better decision making and faster time to value.  

Join this demo to learn how:  
--Data owners can use Snowflake Internal Marketplace to create, document and share secure data products  
--Data consumers can discover, request and access data products with ease 
--Governance policies ensure compliance and access control for data products at all times, even across cloud regions and platforms

Data Mesh Demo: How to Build and Govern Your Data Products in Snowflake

In today’s complex business environment, enterprise AI agents must navigate intricate workflows and decision-making processes, requiring precise access to both structured and unstructured data.

Hear from Thomson Reuters on how they improved decision-making accuracy by building unstructured data preprocessing pipelines, a semantic model for accurate text-to-SQL and AI observability with Snowflake.

Learn how enterprises can leverage these capabilities to enhance decision-making and operational efficiency at scale.

Building Enterprise-Grade AI Agents with Snowflake Cortex AI

As enterprises continue to navigate the complexities of AI-driven transformation, the challenge of integrating structured and unstructured data into scalable, intelligent workflows has never been more critical. In this webinar, veteran technology journalist John K. Waters talks with industry expert Harshal Pimpalkhute about the evolving landscape of AI data agents and how organizations can prepare their data to maximize AI-driven insights.

Among other things, they will discuss the fundamental steps required to process, manage, and govern unstructured data—such as documents, text, and images—while ensuring seamless integration with structured datasets. Attendees will gain insights into best practices for retrieval-augmented generation (RAG), text-to-SQL transformations, vector stores, and semantic modeling, key enablers of AI-driven decision-making.

Join us as we dive into strategies for improving data accuracy, orchestration, and retrieval services, ultimately enabling AI agents to act as powerful intermediaries between business users and enterprise data. Whether you are a data engineer, CIO, or AI strategist, this session will provide actionable insights to elevate your AI data strategy.

Unlocking AI-Powered Insights: Preparing Your Enterprise Data for AI Agents

Generative AI is transforming how enterprises extract insights from sensitive data like documents and images — but cost and performance remain key challenges. Organizations are seeing real ROI by choosing platforms that support both large-scale batch NLP inference and real-time conversational AI.

In this webinar, Snowflake AI experts will share insights and demos on how to:

Develop high-quality, conversational apps faster for self-service analytics
Optimize natural language processing (NLP) pipeline performance with cost-effective LLM batch inference
Serve open-source LLMs and custom embedding models for inference with managed GPUs

Don’t miss this opportunity to see how Snowflake makes gen AI easy, efficient and enterprise-ready with built-in trust.

Gen AI with Snowflake

Join Snowflake’s leading experts as they discuss the AI-accelerated future. They will share predictions about taking LLMs from inspiration to real value, adapting to an AI-powered workforce, securing AI in the enterprise, and more.

Regardless of where you are in your data strategy, we have a number of conversations to help you get value from your data today and achieve generative AI preparedness tomorrow.

This webinar covers upcoming trends across:

- Observability
- Autonomous agents
- Governance
- And more in the age of gen AI

Prepare yourself and your team to tackle your 2025 data initiatives.

AI + Data Predictions 2025: How Operationalizing AI Will Drive Technical Advances and Leadership Challenges

Processing geospatial data is often challenging because it comes from diverse sources, is stored in various formats, and uses different coordinate systems to understand spatial relationships between various features. Many companies struggle to activate their geospatial data into insights. Snowflake can support advanced geospatial processing with native SQL functions and Python UDFs.

Join us in this webinar to learn how you can

- Ingest and process geospatial data from GeoTiffs and Shapefiles
- Read geospatial data using Python UDFs
- Join different sources using nearest neighbor approaches
- Handle larger images efficiently with techniques like resampling and filtering
- Visualize geospatial data using Streamlit, QGIS, and H3 functions
 

By the end of this webinar, you will understand how to prepare data from satellite imagery and shapefiles for machine learning models and advanced spatial analysis within Snowflake.

Best Practices for Processing and Visualizing Geospatial Data in Snowflake

Data engineering is one of the fastest growing jobs in the United States. In fact, in many organizations data engineers are essential for guiding data strategy and pipeline optimization to help businesses leverage their data for increased profitability and growth. 

If you’re a data engineer looking to make the right decisions about data strategies and tools for your organization, join our live webinar, as we discuss 10 best practices for data engineering.

This webinar will cover:  

- Tips for streamlining and operationalizing pipeline development

- The benefits of using data streaming instead of batch ingestion

- The types of tools to invest in

- Skills to help your organization harness the power of data

10 Best Practices for Data Engineers

data engineering

Apache Kafka

data pipelines

continuous data ingestion

SQL server

Data Architecture

Data Analytics

Data Management

Analytics

Are you an IT service management professional interested in developing your knowledge and improving your job performance? Join the IT service management community to access the latest updates from industry experts. Learn and share insights related to IT service management (ITSM) including topics such as the service desk, service catalog, problem and incident management, ITIL v4 and more. Engage with industry experts on current best practices and participate in active discussions that address the needs and challenges of the ITSM community.

IT Service Management

Cloud computing has exploded over the past few years, delivering a previously unimagined level of workplace mobility and flexibility. The cloud computing community on BrightTALK is made up of thousands of engaged professionals learning from the latest cloud computing research and resources. Join the community to expand your cloud computing knowledge and have your questions answered in live sessions with industry experts and vendor representatives.

Cloud Computing

Increasing expectations for good governance, effective risk management and complex demands for corporate compliance are presenting a growing challenge for organizations of all sizes. Join industry thought leaders as they provide you with practical advice on how to implement successful risk and compliance management strategies across your organization. Browse risk management resources in the form of interactive webinars and videos and ask questions of expert GRC professionals.

IT Governance, Risk and Compliance

The data center management community focuses on the holistic management and optimization of the data center. From technologies such as virtualization and cloud computing to data center design, colocation, energy efficiency and monitoring, the BrightTALK data center management community provides the most up-to-date and engaging content from industry experts to better your infrastructure and operations. Engage with a community of your peers and industry experts by asking questions, rating presentations and participating in polls during webinars, all while you gain insight that will help you transform your infrastructure into a next generation data center.

Data Center Management

The application development community features top thought leadership focusing on optimal practices in software development, SDLC methodology, mobile app development and application development platforms and tools. Join top software engineers and coders as they cover emerging trends in everything from enterprise app development to developing for mobile platforms such as Android and iOS.

Application Development

Practicing business intelligence allows your company to transform raw data into sets of insights for targeted business growth. The business intelligence and analytics community on BrightTALK is made up of thousands of data scientists, database administrators, business analysts and other data professionals. Find relevant webinars and videos on business analytics, business intelligence, data analysis and more presented by recognized thought leaders. Join the conversation by participating in live webinars and round table discussions.

Business Intelligence and Analytics

The IT project management community on BrightTALK includes thousands of IT project and portfolio management professionals. Find relevant webinars and videos on agile methodologies, scrum strategy, project management processes and more. Attend live webinars or view on demand content presented by recognized thought leaders in the IT project management industry.

IT Project Management

Welcome to the big data and data management community on BrightTALK. Join thousands of data quality engineers, data scientists, database administrators and other professionals to find more information about the hottest topics affecting your data. Subscribe now to learn about efficiently storing, optimizing a complex infrastructure, developing governing policies, ensuring data quality and analyzing data to make better informed decisions. Join the conversation by watching live and on-demand webinars and take the opportunity to interact with top experts and thought leaders in the field.

Big Data and Data Management

Get powerful marketing data and analytics insights from influential experts. Connect with thought leaders and colleagues to get the most up-to-date knowledge on the analytics that quantify your investments.

Data and Analytics

As an IT professional, many of the problems you face are multifaceted, complex and don’t lend themselves to simple solutions. The information technology community features useful and free information technology resources. Join to browse thousands of videos and webinars on ITIL best practices, IT security strategy and more presented by leading CTOs, CIOs and other technology experts.

Information Technology

Get powerful emerging technology insights from influential experts. Connect with thought leaders and colleagues to get the most up-to-date knowledge on technologies like 3D printing, pervasive robotics, natural interfaces, connected everything and cognitive systems.

Emerging Technologies

The research and development community on BrightTALK brings together leading professionals in healthcare, technology and manufacturing to present on current topics in their fields. From big data in healthcare to online forensics training and medical research practices, the community is an exceptional resource for those looking to expand their knowledge. Be part of the conversation by joining the community and participating in interactive live webinars.

Research and Development

The accounting and tax community on BrightTALK features relevant content around tools and strategies for tax planning, internal controls, auditing and more. Join the conversation by engaging with accounting and tax thought leaders and accountants and asking questions during live webinars and round table discussions on tax and auditing topics.

Accounting and Tax

The accounting and finance community on BrightTALK features presentations from leading accountants and finance professionals. The accounting community includes tax planning strategies, QuickBooks webinars and other accounting webinars, while the finance community features presentations on financial capital regulations and forensic data analytics. Join the conversation by attending live financial and accounting presentations and connecting with industry thought leaders.

Finance

Get powerful insights to run your business. CEOs, CFOs, CMOs, COOs, CTOs and CIOs are among the leaders in this community who connect with peers and experts to grow their businesses.

CxO Strategy

In the business management community, commercial experts will be sharing webinars and videos on an array of subjects. From sales and marketing, to finance, productivity and growth, this content will help you stay up-to-date with the current economic environment and provide timely management insight to drive business growth.

10 Best Practices for Data Engineers

Presented by

About this talk

Snowflake