Program  

Learning Paths
Courses
Location
Corporate
Our Students
Resources
Bootcamp Programs
Short Courses
Portfolio Courses
Bootcamp Programs

Launch your career in Data and AI through our bootcamp programs

  • Industry-leading curriculum
  • Real portfolio/industry projects
  • Career support program
  • Both Full-time & Part-time options.
View Full Catalogue   >
Data Science & Big Data
Full-time   >
Part-time   >
Data Engineering   >

Become a data engineer by learning how to build end-to-end data pipelines

 
Data Analyst (BI)   >

Become a data analyst through building hands-on data/business use cases

Machine Learning Engineering   >

Become an AI/ML engineer by getting specialized in deep learning, computer vision, NLP, and MLOps

DevOps Engineering   >

Become a DevOps Engineer by learning AWS, Docker, Kubernetes, IaaS, IaC (Terraform), and CI/CD

Short Courses

Improve your data & AI skills through self-paced and instructor-led courses

  • Industry-leading curriculum
  • Portfolio projects
  • Part-time flexible schedule
View Full Catalogue   >
CODING
SQL Fundamentals
DATA SCIENCE
Data Science with Python
Applied Machine Learning
Big Data for Data Scientists
AI ENGINEERING
Applied Deep Learning
Portfolio Courses

Learn to build impressive data/AI portfolio projects that get you hired

  • Portfolio project workshops
  • Work on real industry data & AI project
  • Job readiness assessment
  • Career support & job referrals
View Full Catalogue   >
Data Science Client Project   >

Build data strategies and solve ML challenges for real clients

Data Analytics Client Project   >

Help real clients build BI dashboard and tell data stories

Data Engineer Client Project   >

Build end to end data pipelines in the cloud for real clients

Location

Choose to learn at your comfort home or at one of our campuses

ONLINE
North America Online   >
Campuses
Toronto   >
Vancouver   >
Corporate Partners

We’ve partnered with many companies on corporate upskilling, branding events, talent acquisition, as well as consulting services.

Corporate Training   >

AI/Data Transformations with our customized and proven curriculum

DevOps   >
Consulting Services   >

Do you need expert help on data strategies and project implementations? 

Talent Program   >

Hire Data, AI, and Engineering talents from WeCloudData

Our Students

Meet our amazing alumni working in the Data industry

Student Success   >

Read our students’ stories on how WeCloudData have transformed their career

Resources

Check out our events and blog posts to learn and connect with like-minded professionals working in the industry

Blogs   >

Read blogs and updates from our community and alumni

Career Guides   >

Explore different Data Science career paths and how to get started

WeCloudOpen   >

Our free open source courses and workshops gives you the skills and knowledge needed to transform your career in tech

Back to Learning Path
Data Engineering
Big Data Integration & ETL

In the age of big data and data-driven AI, many companies start to realize the importance of establishing data engineering best practices. As a result, the demand for data engineering has been growing rapidly. Currently there is a huge supply and demand mismatch in the talent market. One reason for the imbalance is that modern data engineering requires new tools/skills and traditional learning environments such as universities, colleges, and bootcamps don’t keep up with the trends. Another reason is that data engineering is hard to teach! Curriculums need to be extremely hands-on, and it requires very seasoned instructors who work in the fields to teach in the most practical way.

At a Glance
Advanced
What you will learn

In this 12-week part-time course you will learn how to build complex data pipelines for various real-world use cases. The core skills you will develop in this course include: Docker and Kubernetes, Spark, Spark Streaming, Delta Lake, Data Lakehouse, Data Pipelines, Data Ingestioon, Data Integration, Data Governance.

Summary
Spark, Kubernetes, Airflow, AWS, GCP
Learn to think like a data engineer
Learn and practice through projects
Weekly Leetcode practice and solution sharing

Part-time

Online Live

12 weeks

Register

About the Program

Data engineers are usually harder to train and source because the program needs to be very practical/hands-on and there is not much theory to teach. The open-source communities are also pushing out new tools and platforms on a regular basis which makes teaching data engineering challenging because materials need to be updated rapidly to keep up with the latest trends. At WeCloudData, we have heard from many hiring managers and recruiting agencies say that while the demand for data engineers is great, data engineer talents are even harder to find compared to data scientists.

The Big Data Integration & ETL course is a sequel to the Data Engineering Fundamentals course. In 12 weeks, you will learn how to apply data engineering skills to solve data migration, integration, real-time streaming data processing, as well as data governance, and big data. Completing this course will give you the skills and confidence you need to get a modern data engineer job.

for those who want to
  • Data Engineers from traditional ETL background who want to acquire modern DE skills
  • Developers and Software Engineers who are interested in building data applications
  • IT Professionals who want to explore new paths
  • Data Scientists who want to switch to the engineering world

Speak to our advisor

Our Program Advisor can answer all your questions and help you pick a program that best suits your need. Please fill in your information below and we will contact you.

You can also contact us at info@weclouddata.com or (647) 588-4206

"*" indicates required fields

Name*
This field is for validation purposes and should be left unchanged.

What you will learn

  • Become a better problem solver
    Most online courses will teach specific tools. But knowing 20+ tools doesn't mean you can solve real data problems. We strongly believe that problem-solving skills are the most essential for a successful career in data engineering. Therefore, we've designed a very unique approach to teaching learners how to solve tough problems and become independent thinkers.
    • Build a strong foundation in DE concepts and know the right questions to ask
    • Become a Google/Stackoverflow pro. Know the most relevant keywords to use for problem solving
    • Develop new skills in how to approach new projects and problems
    • Learn modern tools and platforms and know how to put them to work together
    • Become a better team player through group assignments, projects, and client projects
  • Learn Docker and Containerization
    While docker and kubernetes are containerization and orchestration tools that DevOps engineers and infrastructure engineers will work on, data engineers also need to have a good understanding of it. Whether it's setting up airflow servers using docker compose, or running spark jobs on kubernetes, docker is an essential tool that data engineers need to be comfortable with.
    • Learn how docker containers work
    • Know how to deploy Flask apps using docker and docker compose
    • Understand the basics of kubernetes and container orchestration
    • Build hands-on experience with deploying airflow and submit airflow jobs using docker on Kubernetes
  • Become experienced in AWS and GCP
    Cloud computing plays a more and more important role in current days. Applications and services are typically run on servers, which are comprised of CPU-processor, RAM-memory and storage-HDD, SSD. Instead of owning and provisioning servers in the on-prem data center, you could RENT the compute power and move it into the cloud. This means cloud computing is an on-demand delivery of computing power, database storage, applications and other IT resources through cloud services platforms via the internet. Cloud providers(Amazon Web Services-AWS, Google Cloud Platform(GCP), Microsoft Azure) provide rapid access to flexible and low-cost IT resources.
    • Work with several data engineering services on AWS and GCP
    • Hands-on experience with setting up an end-to-end pipeline across multiple services on AWS and GCP
    • Learn how to deploy pipelines in the cloud
  • Build ETL/ELT data pipelines using Apache Airflow
    Apache Airflow is an open-source workflow management platform to programmatically author, schedule, and monitor workflows. It is one of the most robust platforms used by Data Engineers for orchestrating workflows or pipelines. You can easily visualize your data pipelines' dependencies, progress, logs, code, trigger tasks, and success status.
    • Become an advanced Apache Airflow user
    • Hands-on experience with automating your data pipeline using Apache Airflow
    • Develop your ability to maintain and trouble-shoot an Airflow pipeline
  • Data Ingestion: NiFi, Kafka, Kinesis
    Data Ingestion is the first step of a big data processing system and it is essential for the Data ETL pipeline. The data ingestion layer is the backbone of any analytics architecture. Downstream reporting and analytics systems rely on consistent and accessible data. As a Data Engineer, Apache NiFi, Apache Kafka, AWS Kinesis are important and commonly used tools for Data Ingestion.
    • Learn different Data Ingestion tools
    • Ability to process data from different sources using the Data Ingestion tools
    • Hands-on experiences with connecting Data Ingestion tools with the downstream tasks
  • NoSQL and Big Data
    What makes data engineering both exciting and challenging in recent years is the shift from the traditional data warehouse to the data lake and even the new data lakehouse approach. In the data warehouse part of the program, we will cover the fundamental concepts of data modelling, star schema, and snowflake schema. Students will learn how to create ER diagrams and implement their own data warehouse. Modern tools and technologies such as BigQuery, Redshift, and Snowflake will be covered as hands-on labs so students get to experience how it works in practice. We will also cover different scenarios where each approach is a better fit for the problem.
    • Learn the basics of NoSQL databases
    • Understand the key features of NoSQL databases
    • Learn how to work with NoSQL databases

Resources

Watch our recorded webinar and learn more about Data Science career and industry insights.
Data Engineering Program Info Session
Data Engineer Career Panel Discussion
Meet Your Faculty: Zheng Xu
Meet Your Faculty: David Tian

Instructors & Guest Speakers

Edwin Guo
Big Data Specialist @ CPPIB
Edwin is an expert Data Engineer and Cloud Architect who is specialized in Distributed Computing, Distributed Storage, Microservice, Cloud(AWS), Machine learning and Blockchain technologies. Skilled in Spark, Apache Kafka, Akka, Cassandra and various popular tools in the open-source landscape, he has had the opportunity to lead many challenging big data projects in the past. Coming from a Software Engineer background with a strong passion for the CS space, Edwin Guo is well versed in productive languages such as Scala, Java, Clojure, Python, and Golang. His past experience covers many domains including telecommunications, stock market price dissemination/portfolio management system, customer loyalty programs, and cybersecurity.
Preetika Bhateja
Data Engineer & Cloud Architect @Google | Data Engineering Instructor @WeCloudData
Preetika works as a data engineer and cloud architect for Google in the Toronto office. Before Google, she worked in AdTech and banking industries in the data space. In her spare time, she Iikes to geek on anything about data, policy, and politics. She’s passionate about teaching and looks forward to working with students in this program.
WeCloudData Data Engineer Bootcamp Assistant Instructor Zheng Xu
Zheng Xu
Data Engineer, The Globe and Mail
Zheng is currently a data engineer at one of the largest Canadian newspaper company where he builds and supports the data application from other teams in the company. He specializes in building data pipelines using Apache Airflow, data integration and ETL/ELT in Snowflake/Redshift, and developing big data applications using Apache Spark on Databricks. He teaches with WeCloudData in the Data Engineering program as an assistant instructor. Topics he covers include Airflow, Cassandra, Spark, and Spark Streaming.
Bhargavi Singh
Senior Data Analyst, Marketing Science @ Critical Mass
Shaohua Zhang
Chief Instructor (WeCloudData) | Head of Data Science (BeamData) | Startup Advisor and Data Coach
Shaohua is the Co-founder and Chief Instructor at WeCloudData. In the past few years, he has trained hundreds of students and helped many of them launch their data science careers. He is also the CEO and Head of Data Science at Beam Data where he works closely with the industry partners on implementing data science projects. Prior to co-founding the Toronto Institute of Data Science and Technology (WeCloudData), he led the data science team at BlackBerry and helped build the data science practice at Kik. He has also worked with big companies such as TD, Canadian Tire, Sunlife, CIBC, Communitech, and MaRS on upskilling their data teams.
Online Learning Platform

Learn anywhere, anytime

Track your learning journey
Watch lecture recordings, work on coding challenges, ask for TA help, and get resume and job support. The learning portal allows you to track your entire learning journey with ease.
Sharpen your coding skills
Leverage our online coding tool to test your knowledge, identify your weaknesses, and improve your Python and SQL coding skills. The LeetCode style live coding challenges will help you get prepared for technical job interviews.

DATA ENGINEERING CAPSTONE PROJECT

Build portfolio projects that impress hiring managers
Learn how to build end-to-end data engineering pipelines and showcase that to hiring managers. Your resume will shine with the projects you work on in this course.
What you will work on
  • Build complex data ingestion pipelines on AWS/GCP
  • Implement Change Data Capture (CDC) and message brokers
  • Implement data lake and Delta Lake using Apache Hudi and Kafka
  • Streaming data analytics using Spark Streaming and Flink
  • Apply data quality assurance best practices
  • Build reusable and testable pipeline code
STUDENT PROJECT DEMO
WeCloudData Data Engineer Student Project Demo 5
WeCloudData Data Engineer Student Project Demo 4

Reviews

What our students are saying

No results found.

Schedule, Tuition & Financing Options

resources

Related Blog Posts

WeCloud News
Toronto Institute of Data Science and Technology, WeCloudData, Launches New Data Engineering Diploma Program
It has FINALLY arrived! Our long awaited Data Engineering diploma program has launched at last. As the leading institute…
by WeCloudData
January 26, 2021
Student Blog
An Introduction To Spark and Its Behavior.
The blog is posted by WeCloudData’s Big Data course student Abhilash Mohapatra. Checklist Followed: Mapreduce, Hadoop and Spark. Spark…
by Student WeCloudData
October 7, 2020

Related Courses

Portfolio Course

Data Engineer Client Project (Career Mentorship)

View Course
WeCloudData

WeCloudData is the leading data science and AI academy. Our blended learning courses have helped thousands of learners and many enterprises make successful leaps in their data journeys.

Sign up for newsletter

"*" indicates required fields

This field is for validation purposes and should be left unchanged.
Programs
Corporate Services
Resources
Company
Let’s Connect!
Facebook-f Twitter Linkedin Instagram Meetup

433 Yonge St, 2nd Floor,

Toronto ON, M5B 1T3

info@weclouddata.com

+1 (647) 588-4206

View our Big Data Integration & ETL course package
View our Big Data Integration & ETL course package

Learn basic skills with our free WeCloudOpen Courses!

Join our free SQL and Python coding courses now and gain the skills and knowledge you need to start your career. 

Learn More