In this entire note series of Python for Data Science, we will only focus on the basic programming aspects essential for analyzing data and extracting meaningful information rather than deep dive into the nitty-gritty of python programming language or machine learning algorithms.
Data Science
It is an interdisciplinary field that combines computer science, statistics, and mathematics to extract, analyze and generate useful insights from data for better business decisions.
Data science analyzes raw data using simple statistical techniques or more complicated and sophisticated machine learning techniques to draw insights from the data. Nonetheless, the key focus of data science is actually deriving insights using whatever techniques that give better results. It is used in many industries to make better business decisions and test models or theories in the sciences.
When we talk about data science, it is assumed that we have a large amount of data for the problem of interest. We will follow a systematic approach, starting from inspecting the data, cleaning, transforming, modeling, analyzing, and at the end, interpreting raw data to derive valuable insights. The detailed explanation of these steps and their importance are as follows:
- The very first step is to bring data into your system from the different data sources. These data sources may be the data from the different organizations or publically available open data sources, or any other data sources, etc.
- The received data could be in multiple formats, or some column data will be corrupted or missing, or any other problem, etc. So, there is a need for data processing & cleaning. A decision will be taken on handling missing data, multiple data formats to unified format for analysis, selection, and manipulation on data using preferred data structures, etc.
- Once the data is being processed and cleaned, using straightforward statistical measures, it is possible to summarize data to a certain extent. For example, to find central tendency using mean, median, mode, or variance of a particular column, etc.
- As summarization of data has certain limitations. The other step is a visualization of data, in which data will be looked at pictorially to get insights about data. This is a creative aspect of data science, where multiple people could visualize the same data in multiple ways. For example, plotting data and seeing the relationship amongst the columns or attributes.
- When there have a large amount of data, the last step is deriving insights that are not readily apparent through visualization or summarization of data. In this case, we use more sophisticated analytics or analysis of data to get insights, and there comes the machine learning tools and techniques.
Data Science using Python
Python is one of the preferred languages for data science as it has a bundle of open-source packages, libraries that provide the functionalities and tools to perform necessary operations such as import data, processing and cleaning, summarization and visualization, and huge supports of sophisticated machine learning algorithms, etc. Also, it is an easy-to-learn and use kind of programming language.
The major advantage of python is that it provides a good ecosystem of robust and varied libraries. Easy to integrate with big data frameworks like Hadoop, Spark, etc. It supports both object-oriented and functional programming paradigms and reasonably fast for designing prototypes.
Its syntax is simple to use and understand; It has many libraries designed for specific data science tasks and provides APIs to connect to all the major cloud platform services.
In the subsequent notes, we will focus on the few specific libraries to perform the above-mentioned operations.
- Python libraries and data structure provide key feature sets which are essential for data science.
- Data manipulation and pre-processing using the ‘pandas’ library offers various functions for data wrangling, manipulation, and summarization.
- For data visualization and plotting, libraries like ‘matplotlib’ and ‘seaborn’ aid in condensing statistical information and help in identifying trends and relationships.
- Machine learning libraries like ‘sci-kit learn‘ offer a bouquet of machine learning algorithms.
Popular tools using in Data Science
There are many tools that are being used in data science for data processing, and analysis are Python, R, Microsoft Excel, SAS, SPSS, etc. For Data exploration and visualization, tools are Tableau, Qlikview, Microsoft Excel. For parallel and distributed computing, in the case of big data are Apache Spark and Apache Hadoop. However, in this note series will use only Python for doing everything.
References
- NPTEL lectures on Introduction to Python for Data Science, IIT Madras.
CITE THIS AS:
“Python for Data Science – Introduction” From NotePub.io – Publish & Share Note! https://notepub.io/notes/programming-languages/python-for-data-science/python-for-data-science/
256 total views, 1 views today
