上QQ阅读APP看书，第一时间看更新

How AWS empowers data scientists

The number of digital data records that are stored on the internet has grown a lot in the last decade. Due to the drop in storage costs, and new sources of digital data, it is predicted that the amount of digital data stored in 2025 will be 163 zettabytes (1,630,000,000,000 terabytes). Moreover, the amount of data that is generated every day is increasing at an alarming pace, with almost 90% of current data only having been generated during the last two years. With more than 3.5 billion people with access to the internet, this data is not only generated by professionals and large companies, but also by each of the 3.5 billion internet users.

Moreover, since companies understand the importance of data, they store all of their transactional data in the hope of analyzing it and uncovering interesting trends that could help their business make important decisions. Financial investors also crave storing and understanding every bit of information they can get about companies, and train their quantitative analysts or quants to make investment decisions.

It is up to the data scientists of the world to analyze this data and find the gems of information embedded in it. In the last decade, the data science team has become one of the most important teams in every organization. When data science teams were first created, most of the data would fit in Microsoft Excel sheets, and the task was to find statistical trends in the data and provide actionable insights to business teams. However, as the amount of data has increased and ML algorithms have become more sophisticated and potent, the scope of data science teams has expanded.

In the following diagram, we can see the three basic skills that a data scientist needs:

The job description for data scientists varies from company to company. However, in general, a data scientist needs the following three crucial skills:

ML: ML algorithms provide tools to analyze and learn from a large amount of data, and generate predictions or recommendations from that data. It is an important tool for analyzing structured data (such as databases) and unstructured data (such as text documents), and inferring actionable insights from them. A data scientist should be an expert in a plethora of ML algorithms and should understand what algorithm should be applied in a given situation. As data scientists have access to a large library of algorithms that can solve a given problem, they should know which algorithms should be used in each situation.
Computer programming: A data scientist should be an adept programmer, able to write code to access various ML and statistical libraries. There are a lot of programming languages, such as Scala, Python, and R, that provide a number of libraries that let us apply ML algorithms on a dataset. Hence, knowledge of such tools helps a data scientist to perform complex tasks within a feasible time frame. This is crucial in a business environment.
Communication: Along with discovering trends in the data and building complex ML models, a data scientist is also tasked with explaining these findings to business teams. Hence, a data scientist must not only possess good communication skills, but also good analytical and visualization skills. This will help them present complex data models in a way that is easily understood by people not familiar with ML. This also helps data scientists to convey their findings to business teams and provide them with guidance on expected outcomes.