I almost called this blog ‘Things I Would Have Loved to Have Known Before Starting Out on a Career in Data Science’. Given the content of this blog, that sentiment remains true. I think the information contained here will be valuable to anyone looking to meaningfully and concretely familiarize themselves with the data science landscape for the purpose of finding a job.
But data science is a rapidly evolving field, fueled by continual technological advancements that create the need for new skills, and/or re-prioritize traditional skills associated with data science jobs. Consequently, data scientists, regardless of whether they are just starting out or not, will have to adapt to stay relevant in this dynamic and competitive industry.
And because of these realities, I ultimately chose my current title, since it better reflects the forward-looking nature of the blog, focusing on the evolving landscape of data science and the importance of staying updated on job trends, acquiring in-demand skills, and understanding emerging technologies to thrive in this rapidly changing industry.
In short, while I would have loved to have known what I was about to tell you when I was just starting out, I still very much need to know this NOW, and if you’re in data science, you NEED to know this too!
The Problem: Too General
So why then was this information unavailable to me when I began in data science? Well, for the same reason, it’s unavailable to me now. Whenever I was researching the data science job market, all my queries were met with generalities – overly broad distinctions, specifications and requirements, but no hard, accurate and current numbers. Even ChatGPT was not able to provide me with this…yet!
The Solution: A Monthly Data Science Jobs Report
So, where did I manage to obtain this more specific information from, if not from the god of Chatbots? What makes it accurate? And just as importantly, what makes it current?
Well, in order to acquire the most up-to-date picture of the data science landscape, a jobs website was scraped; meticulously accumulating all the data science job postings in North America made over the last month with the further intention of continually scraping and monitoring this data.
The scraper collected everything from geographical locations to salaries, required experience, educational qualifications and of course, the specific technical and soft skills that employers are seeking in their ideal data science candidates. After that, I examined the data looking to see if it would signal certain trends, or at least begin to look for those trends, by creating a monthly jobs report to see how the job market was developing.
In the end, ‘the scrape’ yielded a lot of insights. So, without further ado, I’ll share them with you now so that you might eliminate any of your own guesswork and better navigate your way through this rapidly changing field.
The 4 Data Science Positions and What You Need to Know
While there are a lot of things the data showed, I’ll largely confine it to salary, education and skills, while breaking them down across the four major job types in data science – Data Analyst, Data Engineer, Data Scientist and Machine Learning Engineer.
Before that happens, I wanted to provide some overall observations and context.
1. Salaries were difficult to come by with most employers opting not to provide any specifics on that front.
· Only about 39% of jobs posted a listed salary. Ultimately however, that results in 2840 jobs, so a large enough sample size from which to draw some general conclusions
· Also, of the 39% of jobs that posted a salary, the salary for each position was then tiered from 1-4 based on a quantile range to see if there were any distinctions between the higher paying jobs and the lower paying jobs for each position.
· Finally, the salaries throughout this blog will be reflected in US dollars.
2. It will be interesting to see to what degree the demand for the four data science positions shifts over time given the rise of AI and how those affect each position’s skill set. As of now:
· The role of Data Analyst is the most prevalent posting, consisting of 37% of all job postings in North America and 2691 overall for the entire month.
· The demand for Data Scientists and Data Engineering jobs was neck in neck with the former position comprising 23% of job postings and the latter 22%, or in absolute terms 1658 and 1552, respectively.
· Machine Learning Engineering jobs made up the fewest postings at only 1295, or 18% of jobs, however, as we shall soon see, they were some of the most lucrative. Moreover, given the rise in AI, I suspect this number has already increased from previous months and will continue to do so as larger companies try to leverage the amazing capabilities of Generative AI. That said, I would like to keep a close watch on this position in the coming months and years.
3. Ultimately, I think the news is very good in terms of early entry level positions and minimum educational requirements, and equally good when examining the skill requirements of lower paying and higher paying jobs. The higher paying jobs, for the most part, still sought many of the skill sets for ‘lower’ paying jobs. Consequently, the lower paying jobs will share a lot in common with higher paying jobs, and job seekers should focus on those ‘core’ skills.
4. However, it does seem as though the higher paying positions are reflecting the trend toward one of two things – big data and deep learning. So, if you were looking for upward economic mobility, you might do well to upgrade or enhance your skills around those two things. Below you can see the most in demand skills/tools for each position
Ok. Let’s go into these jobs more concretely.
1. Data Analyst: Uncovering Insights from Data
Job Description: Data analysts are responsible for collecting, cleaning, and interpreting data to uncover valuable insights and trends. They utilize statistical techniques and data visualization tools to present findings to stakeholders, helping organizations make data-driven decisions.
Education Level: A bachelor’s degree in fields like computer science, mathematics, statistics, or a related field is typically required for entry-level data analyst positions. Some roles may require a master’s degree for more advanced analytical positions.
Key Takeaway from the Data:
· This role, in addition to a Data Engineering position, (as we will see), provides the least restrictive barrier of entry in terms of minimum educational requirements. A large percentage of job postings in this field required only a bachelor’s degree.
o 80% of all postings asked for a bachelor’s degree, while 20% asked for a masters and only 0.4% required a PhD.
o These numbers pretty much remained the same regardless of the minimum amount of experience sought. That is, job postings requiring greater years of experience did not translate to higher educational requirements.
Salary Range: On average, it is said that entry-level data analysts can earn around $55,000 to $70,000 per year, while experienced professionals can make upwards of $100,000 per year.
The following is a breakdown of how salary was categorized based on quantile range:
· Tier 1 (75%+): $90,212 or higher
· Tier 2 (50%-75%): $72,800-$90,212
· Tier 3 (25%-%50): $60,000-$72,800
· Tier 4 (0%-25%): $60,000 or less
Key Insight from the Data:
· The scraped results conform to what is thought of as the typical salary range for Data Analysts given that the median salary was $72,800 and the average was calculated to be $77,144
· The presence of some rather large outlier values looks to have had at least some impact on that average, with one posted salary being over 250,000!
· There was not that much difference in salary between Data Analyst jobs requiring a Bachelor’s Degree as a minimum requirement as compared to those asking for a Master’s Degree with posted salaries of $77,460 and $80,806, respectively.
Technical Tools/Skills: Regardless of tier, there was a strong demand for a similar core skillset:
· Proficiency in SQL and Excel for data querying and manipulation.
· Basic programming skills in Python or R for data analysis tasks
· Knowledge of data visualization tools like Tableau, and Power BI, for creating compelling visualizations
· Looking at how these skills break down among the higher paying jobs compared to lower paying ones…
Key TakeAway from the Data:
· It looks as though Data Analysts who are more involved with SQL and Python are enjoying higher salaries and are less engaged with Excel as compared to their lower paid counterparts.
· There is also a considerable discrepancy with respect to big data related tools as tier 1 Data Analyst jobs are being asked to engage with Cloud, AWS and Azure, while tier 4 jobs are not for the most part.
· The same can also be said for database management systems like Oracle SQL Server
2. Data Engineer: Building Robust Data Infrastructures
Job Description: Data engineers are responsible for designing, constructing, and maintaining data pipelines and databases. They ensure data quality, scalability, and accessibility, allowing data scientists and analysts to access clean and reliable data.
Education Level: Data engineers typically hold a bachelor’s degree in computer science, software engineering, or a related field. There are roles however that may require a higher level of education. The need for higher levels of education beyond a bachelor’s degree may increase slightly with jobs requiring more experience, however.
Key Insights from the Data:
· This job, more than any other, provides the lowest barrier to entry in terms of minimum educational requirements.
o 85% of all data engineering posts required only a bachelor’s degree, while specifying a master’s degree as a minimum educational requirement in only 14% of postings – the lowest percentage across all the data science job postings.
Salary Range: Entry-level data engineers are said to earn around $70,000 to $90,000 per year, while senior data engineers can earn over $120,000 per year. According to the available jobs, the average salary was $111,453.
The following is a breakdown of how the jobs broke down by category, (tiers) based on the quantile range of their salary:
· Tier 1 (75%+): $130,000 or higher
· Tier 2 (50%-75%): $106,000-$130,000
· Tier 3 (25%-%50): $87,757-$106,000
· Tier 4 (0%-25%): $87,757 or less
Key Insight from the Data:
· Once again the scraped salaries here essentially mirror the expected salary range with a median salary of $106,000 and an average salary of $111,453, according to the data
· There were fewer outliers here, however the highest posted salary was nearly $290,000!!
· Job postings for Data Engineers requiring a Bachelor’s Degree as a minimum requirement had an average salary of $109,879, while postings that required a Master’s Degree as a minimum requirement had an average salary of $115,449.
· Job postings for Data Engineers requiring a Bachelor’s Degree had an average salary of $109,879, while postings that require a Master’s Degree as a minimum requirement
Technical Skills: Regardless of tier, we see the demand for big data skills in addition to some other core skills:
· Strong programming skills in languages like Python and Java,
· Expertise in data storage technologies, particularly SQL
· Knowledge of the cloud platforms like AWS and Azure for building scalable data infrastructures
· Spark to deal with large data sets
· Looking at how these skills break down among the higher paying jobs compared to lower paying ones…
Key Insights from the Data:
· The results here are interesting because it seems that tier 4 data engineering jobs are engaging with the same skills and tools to the same degree or even higher, with the exception of some key tools like AWS, Spark, Snowflake and Airflow
· Ultimately these skills confirm the demand for big data knowledge in companies. So, as the volume of data grows, data engineers will be continually challenged to build scalable and efficient data infrastructures that can handle the influx of data from various sources.
3. Data Scientist: Transforming Data into Insights
Job Description: Data scientists use advanced statistical and machine learning techniques to extract insights from data. They design and develop predictive models to solve complex business problems and provide valuable insights to support decision-making.
Education Level: It’s thought that Data Scientists typically hold a master’s or doctoral degree in fields like computer science, statistics, mathematics, or a related field. This holds true to some extent, but the field is very much accessible to those who do not have a master’s degree or higher.
Key Insights from the Data:
· Looking at data science positions 33% of jobs asked for a master’s degree, the highest ask of any of the four positions, Machine Learning Engineers included.
· While this looks daunting, keep in mind that more than half (57%) of job postings designated a bachelor’s degree as a minimum requirement.
Salary Range: Entry-level data scientists can earn around $80,000 to $100,000 per year, while experienced data scientists can make over $150,000 per year.
The following is a breakdown of how the jobs broke down by category, (tiers) based on the quantile range of their salary:
· Tier 1 (75%+): $137,325 or higher
· Tier 2 (50%-75%): $110,000-$137,325
· Tier 3 (25%-%50): $85,058-$110,000
· Tier 4 (0%-25%): $85,058 or less
Key Insights from the Data:
· Considering all jobs, the average salary was $115,111, which is slightly higher than the average for data engineers and a median salary of $110,000.
· There are some exceptional outlier values here with two positions with a number of positions posting salaries above $215,000.
· There was a significant difference in salary when it came to the Data Scientist position and the minimum educational requirement – certainly more than the two previous jobs.
- Data Scientists with a Bachelor’s Degree earned on average $112,036, while those with a Master’s Degree earned $129,116.
Technical Skills: Once again, we see a core skill set that is very much in demand throughout all the data science jobs.
· Proficiency in programming languages like Python or R for data analysis and model development.
· Knowledge of SQL to efficiently extract, manipulate, and analyze data from databases
· Familiarity with data visualization tools – particularly Tableau – for communicating insights
· Employers are also asking for AWS and Deep Learning
· Looking at how these skills break down among the higher paying jobs compared to lower paying ones…
Key Insights from the Data:
· It looks as if companies are increasingly considering both deep learning and big data and requesting familiarity with the technologies surrounding them.
· The skillsets around these two areas are particularly pronounced with tier 1 jobs listing deep learning tools like Pytorch and Tensorflow, in addition to big data tools like Spark and AWS and other Cloud related tools and platforms.
· It seems companies are prioritizing the power of big data and deep learning with skills that may overlap significantly with machine learning engineers, a position to which we will now look at….
4. Machine Learning Engineer: Powering Intelligent Systems
Job Description: Machine learning engineers are responsible for designing, building, and deploying machine learning models and systems. They work closely with data scientists and data engineers to develop scalable machine learning solutions that power intelligent systems.
Education Level: It is thought that Machine learning engineers typically hold a master’s or doctoral degree in fields like computer science, machine learning, artificial intelligence, or a related field and that a strong educational background in machine learning and computer science is essential. The data revealed something different when it came to minimum educational requirements…
Key Insights into the Data:
· This field remains very open to individuals who do not have a master’s degree
- With 57.2% of positions asking only for a Bachelor’s Degree as a minimum, it is at least on par with Data Science positions that require a Bachelor’s degree 56.6% of the time as a minimum educational requirement.
· Aside from data scientists, these positions also had the second highest request rate for master’s degrees as minimum level of education.
· From all the job postings on this position, there was a surprisingly low request rate for PhDs (13.4%).
Salary Range: According to the internet, Machine learning engineers typically earn salaries comparable to data scientists. Entry-level machine learning engineers may earn around $80,000 to $100,000 per year, while experienced professionals can earn over $150,000 per year. The scraped data revealed something different, particularly when it came to entry level positions…
The following is a breakdown of how the jobs broke down by category, (tiers) based on the quantile range of their salary:
· Tier 1 (75%+): $166,400 or higher
· Tier 2 (50%-75%): $135,200-$166,400
· Tier 3 (25%-%50): $112,033-$135,200
· Tier 4 (0%-25%): $112,033 or less
Key Insights from the Data:
· This is the most lucrative field of data science, yielding the highest averages, with salary figures well above what a typical Data Scientist would make.
- The average salary was $137,749 and the median salary coming in at $135,200
- Entry level positions, thought to be typically tier 4 positions, had an upper range of 112,033
· There were not that many outliers, not like the ones that were seen in the Data Scientist positions, so we can consider these salary figures to be fairly reliable.
· Interestingly enough, there was not a major difference between those positions asking for a Bachelor’s Degree and those asking for a Master’s Degree as a minimum educational requirement – $140,533 and $142,876, respectively
Technical Skills: To no surprise, we see a heavily favored disposition towards big data and AI/deep learning here. Regardless of the job tier, employers are looking for:
- Proficiency in programming languages like Python or Java for model development and deployment.
- Experience with deep learning frameworks like TensorFlow or PyTorch.
- Familiarity with cloud platforms for scalable model deployment like AWS as well as Kubernetes and Spark
· Looking at how these skills break down among the higher paying jobs compared to lower paying ones…
Key Insights from the Data:
· The theme across all job types holds true here too. Deep Learning and Big Data are the most sought after skills for those in tier 1 jobs for machine learning engineers.
· It is interesting to note, here too, both tiers are engaging similar tools, near or at par with each other. It seems that, at least initially anyway, this is good news for tier 4 positions, and it may be just a matter of time and experience before they ascend to the higher paying positions.
Conclusion
The future of data science holds tremendous opportunities for skilled professionals in data analysis, engineering, science, and machine learning. Each role plays a crucial part in transforming raw data into actionable insights that drive innovation and support data-driven decision-making. As the industry evolves, the demand for these roles will continue to grow, along with the need for a diverse set of technical skills.
By staying updated with the latest trends and continuously honing technical expertise, data professionals can thrive in the dynamic and competitive landscape of data science. The journey to a rewarding and impactful career in data science begins with a passion for data and a commitment to embrace the ever-evolving nature of the field. Let the future of data science be your guide on this exhilarating and transformative path.
