What Made Me Interested in Data Engineering?
My curiosity started with Big Data, Data Analytics, and Data Science since it was all over the news. As I delved deeper, I discovered the role of Data Engineering and felt that it fit better with my previous experience in software engineering and more aligned with my interests. Finally, from a very practical standpoint, I also considered the good salaries, tight labour market, and plentiful opportunities for the foreseeable future.
Before the Bootcamp
In the months before the Bootcamp, I did my best to brush up on the fundamental skills that we would be using regularly. I started with Python and SQL, then moved on to AWS cloud computing. I studied deep enough to get comfortable so that I wouldn’t get too overwhelmed when the fast-paced teaching came. It was a very good decision as it helped out a lot later.
In my particular case, I had worked in software engineering years ago but left the industry entirely for a number of years. So in regards to coding, I was rusty at best, if not completely obsolete. General software concepts are all similar so learning Python and cloud computing was not difficult. However, SQL, Database concepts, Docker, and Kubernetes were completely new and I found them quite tricky. I only wish I had pre-studied these subjects more beforehand.
Learning Experience
As with any Bootcamp, a tremendous amount of learning is squeezed into a short time. That’s by far the biggest challenge. New concepts take time to really wrap your head around and digest so pre-studying is highly recommended. The instructors are very knowledgeable, sharp and open to discussion, but you can only fit so much material in a 3-hour lecture. The rest is on your own time using additional resources for self-learning – which is not always easy if you’re used to being spoon-fed in a traditional classroom setting – but we luckily live in a world where virtually any question can be answered with Google. It’s important to understand that most of your learning will be done outside of lectures and labs.
Projects
We had both mid-term and final projects. Both were data pipeline implementations on the cloud that required us to use all the common industry tools from data ingestion to data dashboard visualization, plus everything in between. These were invaluable learning experiences because there is simply no way to learn without diving in and doing it. The general framework was provided to us and we were required to implement additional capability and functionality. You were allowed the flexibility of taking it as far as you wanted to go.
Each project had a deadline and required a presentation to the class and instructors. The pressure to perform was real but it was a good kind of stress that pushed you to learn quickly.
Job Search and Interviews
It was both sad and exciting to finish up the Bootcamp but this was the real final test. I mainly used LinkedIn, Indeed and Glassdoor to search for jobs, which fortunately were relatively plentiful. The school’s founder(s) have connections to hiring managers and recruiters in the industry so they are able to provide some job referrals too. These referrals would land you an interview much more easily, but it’s still up to you to showcase yourself and convince them to hire you.
Job hunting is a stressful process as you need to stay positive, think optimistic, and project confidence, even when you have doubts. I applied to plenty of jobs and received plenty of rejections but I strictly viewed it as a numbers game. As expected, only a minority of applications ever resulted in interviews. At the same time, knowing there was (and still is) a severe lack of Data Engineers in the market allowed me to expand to include positions that I may not have been fully qualified for, which surely lowered my application-to-interview ratio but resulted in a higher total number of interviews. Inevitably, I bombed some interviews but brushed it off and used it as a learning experience for next time. In the end, I applied to approx 130 jobs and received around 20 interviews
I believe it was the WeCloudData corporate projects that made all the difference. It’s your past working experience that prospective employers are most interested in. With only academic projects under your belt, there’s a certain awkwardness because the interviewer can’t ask his/her usual questions and nor can you talk about the projects as if they were real. WeCloudData corporate projects were real-life projects with real-life responsibilities. Most importantly, they empower you with a confidence boost and let the conversation flow easily such that the interviewer sees you as an existing professional working engineer instead of a freshly graduated student.
In summary, the job-hunting process has its frustrations but that’s a part of the whole package. We are fortunate enough to be in a tight job market (for the foreseeable future) which gave me the opportunity to fail multiple times, adjust my approach, still keep getting interviews, and eventually ace a few to end up with multiple offers. And that was all crammed into 1.5 months. Even if the market cooled off and it required double to triple the time, i.e. 3 – 4.5 months, that’s still a very good outcome.
Suggestions for New Students
If I was to do it all over again, I would look at the curriculum and understand every topic at a high level first, and if time permits keep going deeper. Depending on your background, some, many, or all of the topics may be completely new and you need to give yourself time to digest them.
Pay particular attention to both Python and SQL (you can never have enough), and to a slightly lesser extent, Amazon AWS or Google GCP. There are tons of free lectures and tutorials available on youtube. Or there are higher-quality paid learning resources too. Also read up on databases – different types, design, pros/cons, etc.
The majority of your learning will come from doing the academic projects and the corporate projects. These are critical – possibly even more so than academic studying – it’s the only way to properly learn the skills. You’ll undoubtedly be talking about these projects during your future job interviews so put 100% effort into them.
Reflecting on My Experience
Looking back, I think it was a good decision to choose WeCloudData. Maybe I could have done it without them but would have taken 5 times longer with a high chance of giving up along the way. And that’s just the academic portion. Finding a job would also have taken 5 times longer, especially given that I would be lacking the corporate project experience and the resulting confidence boost.
There’s a big debate about Bootcamps in general. There are plenty of self-taught people out there, especially on youtube, who promote it as a way to save money. It’s valid but I believe that many people (if not most) don’t have that iron-willed determination to power through a complicated and comprehensive curriculum like Data Engineering by themselves. Most of us need classmates, instructors, and TAs helping to push us along. You also have to consider the opportunity cost – the time between starting the Bootcamp to starting a new job for me was roughly 7 months. I highly doubt most people could accomplish that in the same amount of time going the self-study route. Note that other more-experienced classmates landed jobs even earlier too. Looking at the big picture, getting yourself to market outweighs the savings.
I would also note that the not-so-small tuition fee also psychologically commits you to put in maximum effort and gives you a nagging fear of wasting that chunk of money. It further serves to push you toward your ultimate goal.
It was a lot of hard work but it paid off with the final outcome. And I’m grateful that WeCloudData helped at each step to get me there in such a short time.
