Sebastian Gomez
Software Developer Trend Analysis
Note (updated on June 24, 2026). This article is a historical snapshot. The dataset only reaches the 2019 Stack Overflow survey, so the projections presented here are now well past their forecast horizon and can be compared against what actually happened between 2020 and 2025. Also, the tool Google Data Studio was rebranded as Looker Studio in October 2022, so we use that name throughout the text. The links to external resources (repository, report, and Gist) date from 2022 and may no longer resolve; in particular, the report URLs migrated from the datastudio.google.com domain to the new lookerstudio.google.com. It is worth verifying them before use.
This document aims to describe and detail a trend analysis of the technology job market, in which the main goal is to analyze factors such as historical changes and compensation projections for workers in this sector. It also aims to analyze the historical data and produce projections about the supply and demand of the programming languages in use between 2011 and 2019.
Introduction
This analysis was carried out using data from Stack Overflow, one of the most important communities in technology. As tools to manipulate and later analyze the data, we used Google cloud services: BigQuery to process the data through SQL statements, and Looker Studio (formerly Google Data Studio) to chart and analyze the data. Excel was also used as an intermediate program to make small transformations to the data. The trend analysis consisted of eight (8) stages, of which the data transformation took up roughly 70-80% of the total time. These stages are described in more detail in the following sections of this article.
Data extraction
All the data required to carry out this analysis was obtained from the official Stack Overflow website, from which we downloaded the files in comma separated format (CSV), where each file corresponds to an annual survey for the years between 2011 and 2019.
A note on the number of years. The annual surveys for the 2011-2019 period were downloaded, which would amount to nine files; however, the 2015 survey did not include usable compensation data, so that year is omitted from the compensation query. That is why the analysis effectively works with eight years of compensation data (2011-2014 and 2016-2019), while the language analysis does use all nine available years.
These are the datasets used:
- Repository link (2022 link; verify that it still resolves)
Data cleaning
Generally, the kinds of data found in survey databases are not fully structured, complete, or of high information quality. For this reason, a data cleaning stage had to be carried out to remove from the databases all data that did not add any value to this analysis.
This initial cleaning consisted of removing all the columns that did not represent valuable information for the knowledge this work sought to generate, which is basically compensation analysis in the technology job market and trend analysis of programming language usage. Data such as questions about personal preferences, country of residence, influence on company purchases, the type of project the person was working on, among other data, were not taken into account for this analysis.
This initial data cleaning was done with Excel, allowing us to reduce the size of the databases to be analyzed and to move on with cleaner, lighter data to the next stage: data loading.
Loading data into Google BigQuery
Once the data cleaning was complete, removing all the data that was not useful to us, we proceeded to load our eight compensation databases, corresponding to the Stack Overflow surveys, into Google BigQuery, the tool we used to manipulate our data and obtain results that brought us closer to the desired analysis.
Data manipulation
We now enter a fundamental stage for the analysis we want to carry out: data manipulation is the process performed on the data that lets us turn the input data into slightly more aggregated output data, so that we can later turn this data into information and then into knowledge.
Although it may seem like a purely operational process, in the data manipulation stage it is vital that the person responsible for this activity carefully analyze and ask themselves what kind of data they have available and what questions they will pose, then translate those questions into the system to obtain the data with the highest possible added value.
Some of the many questions posed at this stage are detailed below, some with their corresponding Google BigQuery SQL query and the associated Looker Studio chart that answers the question.
What are the most used programming languages among Stack Overflow users across all the years between 2011 and 2019?
SELECT
`trabajofinalsmbd.StackOverflow.WLanguages2011`.Lenguaje,
`trabajofinalsmbd.StackOverflow.WLanguages2011`.Tech AS _2011,
`trabajofinalsmbd.StackOverflow.WLanguages2012`.Tech AS _2012,
`trabajofinalsmbd.StackOverflow.WLanguages2013`.Cantidad AS _2013,
`trabajofinalsmbd.StackOverflow.WLanguages2014`.Cantidad AS _2014,
`trabajofinalsmbd.StackOverflow.WLanguages2015`.Cantidad AS _2015,
`trabajofinalsmbd.StackOverflow.WLanguages2016`.Cantidad AS _2016,
`trabajofinalsmbd.StackOverflow.WLanguages2017`.Cantidad AS _2017,
`trabajofinalsmbd.StackOverflow.WLanguages2018`.Cantidad AS _2018,
`trabajofinalsmbd.StackOverflow.WLanguages2019`.Cantidad AS _2019
FROM
`trabajofinalsmbd.StackOverflow.WLanguages2011`,
`trabajofinalsmbd.StackOverflow.WLanguages2012`,
`trabajofinalsmbd.StackOverflow.WLanguages2013`,
`trabajofinalsmbd.StackOverflow.WLanguages2014`,
`trabajofinalsmbd.StackOverflow.WLanguages2015`,
`trabajofinalsmbd.StackOverflow.WLanguages2016`,
`trabajofinalsmbd.StackOverflow.WLanguages2017`,
`trabajofinalsmbd.StackOverflow.WLanguages2018`,
`trabajofinalsmbd.StackOverflow.WLanguages2019`
WHERE
`trabajofinalsmbd.StackOverflow.WLanguages2011`.Lenguaje = `trabajofinalsmbd.StackOverflow.WLanguages2012`.Lenguaje
AND `trabajofinalsmbd.StackOverflow.WLanguages2011`.Lenguaje = `trabajofinalsmbd.StackOverflow.WLanguages2013`.Lenguaje
AND `trabajofinalsmbd.StackOverflow.WLanguages2011`.Lenguaje = `trabajofinalsmbd.StackOverflow.WLanguages2014`.Lenguaje
AND `trabajofinalsmbd.StackOverflow.WLanguages2011`.Lenguaje = `trabajofinalsmbd.StackOverflow.WLanguages2015`.Lenguaje
AND `trabajofinalsmbd.StackOverflow.WLanguages2011`.Lenguaje = `trabajofinalsmbd.StackOverflow.WLanguages2016`.Lenguaje
AND `trabajofinalsmbd.StackOverflow.WLanguages2011`.Lenguaje = `trabajofinalsmbd.StackOverflow.WLanguages2017`.Lenguaje
AND `trabajofinalsmbd.StackOverflow.WLanguages2011`.Lenguaje = `trabajofinalsmbd.StackOverflow.WLanguages2018`.Lenguaje
AND `trabajofinalsmbd.StackOverflow.WLanguages2011`.Lenguaje = `trabajofinalsmbd.StackOverflow.WLanguages2019`.Lenguaje
ORDER BY _2019 DESCWhat is the minimum and maximum compensation range between 2011 and 2019 according to people's years of experience? Note that this question implicitly contains eight of these same questions applied to a single year between 2011 and 2019. As mentioned earlier, the 2015 survey does not provide usable compensation data, so that year does not appear in the query (it jumps from 2014 to 2016). For convenience, we answer all of these questions with a single SQL query.
SELECT '2011' AS Year, Experience,
SUBSTR(Compensation,3,6) AS CompMin,
SUBSTR(Compensation,3,6) AS CompMax
FROM `trabajofinalsmbd.StackOverflow.SO_2011`
WHERE SUBSTR(Compensation,1,1) LIKE '%<%'
UNION ALL
SELECT '2011', Experience,
SUBSTR(Compensation,3,7),
SUBSTR(Compensation,3,7)
FROM `trabajofinalsmbd.StackOverflow.SO_2011`
WHERE SUBSTR(Compensation,1,1) LIKE '%>%'
UNION ALL
SELECT '2011', Experience,
SUBSTR(Compensation,2,6),
SUBSTR(Compensation,12,6)
FROM `trabajofinalsmbd.StackOverflow.SO_2011`
WHERE LENGTH(Compensation) = 17
UNION ALL
SELECT '2011', Experience,
SUBSTR(Compensation,2,6),
SUBSTR(Compensation,12,7)
FROM `trabajofinalsmbd.StackOverflow.SO_2011`
WHERE LENGTH(Compensation) = 18
UNION ALL
SELECT '2011', Experience,
SUBSTR(Compensation,2,7),
SUBSTR(Compensation,13,7)
FROM `trabajofinalsmbd.StackOverflow.SO_2011`
WHERE LENGTH(Compensation) = 19
UNION ALL
SELECT '2012' AS Year, How_many_years_of_IT_Programming_experience_do_you_have_,
SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,3,6) AS CompMin,
SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,3,6) AS CompMax
FROM `trabajofinalsmbd.StackOverflow.SO_2012`
WHERE SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,1,1) LIKE '%<%'
UNION ALL
SELECT '2012' AS Year, How_many_years_of_IT_Programming_experience_do_you_have_,
SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,3,7),
SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,3,7)
FROM `trabajofinalsmbd.StackOverflow.SO_2012`
WHERE SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,1,1) LIKE '%>%'
UNION ALL
SELECT '2012' AS Year, How_many_years_of_IT_Programming_experience_do_you_have_,
SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,2,6),
SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,12,6)
FROM `trabajofinalsmbd.StackOverflow.SO_2012`
WHERE LENGTH(Including_bonus__what_is_your_annual_compensation_in_USD_) = 17
UNION ALL
SELECT '2012' AS Year, How_many_years_of_IT_Programming_experience_do_you_have_,
SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,2,6),
SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,12,7)
FROM `trabajofinalsmbd.StackOverflow.SO_2012`
WHERE LENGTH(Including_bonus__what_is_your_annual_compensation_in_USD_) = 18
UNION ALL
SELECT '2012' AS Year, How_many_years_of_IT_Programming_experience_do_you_have_,
SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,2,7),
SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,13,7)
FROM `trabajofinalsmbd.StackOverflow.SO_2012`
WHERE LENGTH(Including_bonus__what_is_your_annual_compensation_in_USD_) = 19
UNION ALL
SELECT '2013' AS Year, How_many_years_of_IT_Programming_experience_do_you_have_,
SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,3,6) AS CompMin,
SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,3,6) AS CompMax
FROM `trabajofinalsmbd.StackOverflow.SO_2013`
WHERE SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,1,1) LIKE '%<%'
UNION ALL
SELECT '2013' AS Year, How_many_years_of_IT_Programming_experience_do_you_have_,
SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,3,7),
SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,3,7)
FROM `trabajofinalsmbd.StackOverflow.SO_2013`
WHERE SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,1,1) LIKE '%>%'
UNION ALL
SELECT '2013' AS Year, How_many_years_of_IT_Programming_experience_do_you_have_,
SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,2,6),
SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,12,6)
FROM `trabajofinalsmbd.StackOverflow.SO_2013`
WHERE LENGTH(Including_bonus__what_is_your_annual_compensation_in_USD_) = 17
UNION ALL
SELECT '2013' AS Year, How_many_years_of_IT_Programming_experience_do_you_have_,
SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,2,6),
SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,12,7)
FROM `trabajofinalsmbd.StackOverflow.SO_2013`
WHERE LENGTH(Including_bonus__what_is_your_annual_compensation_in_USD_) = 18
UNION ALL
SELECT '2013' AS Year, How_many_years_of_IT_Programming_experience_do_you_have_,
SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,2,7),
SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,13,7)
FROM `trabajofinalsmbd.StackOverflow.SO_2013`
WHERE LENGTH(Including_bonus__what_is_your_annual_compensation_in_USD_) = 19
UNION ALL
SELECT '2014' AS Year, How_many_years_of_IT_Programming_experience_do_you_have_,
SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,3,6) AS CompMin,
SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,3,6) AS CompMax
FROM `trabajofinalsmbd.StackOverflow.SO_2014`
WHERE SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,1,1) LIKE '%<%'
UNION ALL
SELECT '2014' AS Year, How_many_years_of_IT_Programming_experience_do_you_have_,
SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,3,7),
SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,3,7)
FROM `trabajofinalsmbd.StackOverflow.SO_2014`
WHERE SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,1,1) LIKE '%>%'
UNION ALL
SELECT '2014' AS Year, How_many_years_of_IT_Programming_experience_do_you_have_,
SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,2,6),
SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,12,6)
FROM `trabajofinalsmbd.StackOverflow.SO_2014`
WHERE LENGTH(Including_bonus__what_is_your_annual_compensation_in_USD_) = 17
UNION ALL
SELECT '2014' AS Year, How_many_years_of_IT_Programming_experience_do_you_have_,
SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,2,6),
SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,12,7)
FROM `trabajofinalsmbd.StackOverflow.SO_2014`
WHERE LENGTH(Including_bonus__what_is_your_annual_compensation_in_USD_) = 18
UNION ALL
SELECT '2014' AS Year, How_many_years_of_IT_Programming_experience_do_you_have_,
SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,2,7),
SUBSTR(Including_bonus__what_is_your_annual_compensation_in_USD_,13,7)
FROM `trabajofinalsmbd.StackOverflow.SO_2014`
WHERE LENGTH(Including_bonus__what_is_your_annual_compensation_in_USD_) = 19
UNION ALL
SELECT '2016' AS Year, experience_range,
CAST(salary_midpoint AS String) AS CompMin,
CAST(salary_midpoint AS String) AS CompMax
FROM `trabajofinalsmbd.StackOverflow.SO_2016`
WHERE salary_midpoint IS NOT NULL
UNION ALL
SELECT '2017' AS Year, YearsProgram,
Salary AS CompMin, Salary AS CompMax
FROM `trabajofinalsmbd.StackOverflow.SO_2017`
WHERE Salary IS NOT NULL AND Salary <> 'NA' AND Salary <> '0'
AND SUBSTR(Salary,2,1) NOT LIKE '%.%'
AND SUBSTR(Salary,3,1) NOT LIKE '%.%'
AND LENGTH(Salary) > 3
AND SUBSTR(Salary,4,1) NOT LIKE '%.%'
UNION ALL
SELECT '2018' AS Year, YearsCoding,
ConvertedSalary__USD_Yearly_ AS CompMin,
ConvertedSalary__USD_Yearly_ AS CompMax
FROM `trabajofinalsmbd.StackOverflow.SO_2018`
WHERE ConvertedSalary__USD_Yearly_ IS NOT NULL
AND ConvertedSalary__USD_Yearly_ <> 'NA'
AND LENGTH(ConvertedSalary__USD_Yearly_) > 3
UNION ALL
SELECT '2019' AS Year, YearsCode,
ConvertedComp__usd_Year_ AS CompMin,
ConvertedComp__usd_Year_ AS CompMax
FROM `trabajofinalsmbd.StackOverflow.SO_2019`
WHERE ConvertedComp__usd_Year_ IS NOT NULL
AND ConvertedComp__usd_Year_ <> 'NA'
AND LENGTH(ConvertedComp__usd_Year_) > 3What is the average compensation between 2011 and 2019 according to people's experience? Note that this question differs from the previous one in that it is an evolution of it, with more aggregated data. Here we no longer have the full list of minimum and maximum compensations for all years; instead, we have the average compensation grouped by year and by the person's experience.
SELECT Year, Experience,
CAST(AVG(CompMin+CompMax) AS INT64) AS SalarioPromedio
FROM `trabajofinalsmbd.StackOverflow.Salarios_Experiencia`
GROUP BY Year, Experience
ORDER BY Year DESC, SalarioPromedio DESCGenerating a report in Looker Studio
Once the data manipulation and transformation stage was complete, we proceeded to generate a report in Looker Studio (formerly Google Data Studio) in order to use the data obtained in previous stages and turn it into valuable information that would give us a very broad and at the same time very detailed picture of technology trends between 2011 and 2019.
Below we share the link to the Looker Studio report, where the two factors mentioned earlier were analyzed: compensation and programming language usage.
- Looker Studio report (2022 link; the URLs migrated from `datastudio.google.com` to `lookerstudio.google.com`, verify that it still resolves)
Data prediction
Finally, the main goal of carrying out this analysis was to be able to generate predictions of the data into the future using the data available to us.
For this we used Google Colab, into which the data was imported and an ARIMA model was implemented to carry out this prediction, given that the data we use resembles a time series and that data points over time are autocorrelated, that is, one data point depends on a past one. It is also worth noting that this is very possibly not the best model for this prediction, and that perhaps models such as Holt Winters would be more appropriate.
- Gist link (2022 link; verify that it still resolves)
Conclusions
This analysis showed that, for some years now, the technology job market has been displaying very pronounced trends regarding the programming languages in use. Languages such as JavaScript keep gaining ground, surely because of their application in an ever growing market like the web.
Languages such as SQL started the decade and ended it with substantial participation among the surveyed users; this is due to their great usefulness in manipulating relational databases, their simplicity, and their many uses. In short, SQL has become a basic competency in the technology world. On the other hand, we have languages such as Python, which, although they did not start the decade in the top 3 most used languages, did end the decade in the top 3, and Python has gained more and more traction due to its simplicity, its great scalability, and its countless possible uses, especially regarding Data Science and Artificial Intelligence.
As for the other aspect of the analysis, compensation in the technology sector over the last decade, there are no changes or behaviors that fall outside what was expected. Salaries increase as time passes, perhaps driven by factors such as Industry 4.0. We can also see that people with more years of experience in technology tend to have a higher average compensation than those with less experience; however, this behavior extends to sectors other than technology as well.
On the validity of these conclusions. These trends and projections were computed on data ending in 2019. Anyone reading this article today can compare them against the Stack Overflow surveys from 2020 to 2025 to see how accurate they turned out to be.
Acknowledgements
This article was written together with the student:
- Juan Esteban Arroyave Duque
Students of the master's degree and specialization in predictive analytics at Universidad Nacional, Medellín campus, in the Massive Database Systems course 2020-I.
That's all, I hope this post is useful to you. If you have any questions, do not hesitate to leave me a comment below, and remember that if you liked it you can also share it using the social media links below.
Sebastian Gomez
Creador de contenido principalmente acerca de tecnología.