Shanto Roy

I Explore Research, Tech, Travel, and Business. PhD Student @UHCS.

3 minute read

pandas is the mostly used module while analyzing data from a CSV, JSON, or Excel as the module can convert these data into dataframes and helps to analyze the data.

In this tutorial, we will see some basic usages of this module.

First thing first, let’s import the module

import pandas as pd

Now, let’s take a look at some basic tips and tricks

Series

We can create a series from a list or dictionary where for a list, the index column has index values ($0,1,2,\dots$) and for a dictionary, the index column has key values (for below example, $a,b,c$).

mylist = ['a','b','c']
mynums = [1,2,3]
mydict = {'a':10,'b':20,'c':30}
print(pd.Series(data=mylist))
print(pd.Series(data=mydict))
print(pd.Series(mynums,labels))
# pd.Series(data=mynums,index=labels)

If we use two lists inside the pd.Series method, the first one’s items are put in the data column, and the second list items are put in the index column.

Dataframes

Creating Dataframes

We can simply create a dataframe using the pandas.Dataframe() method. The following example creates a matrix of size $5 \times 4$ with random numbers, and then set index labels from A to E, and column labels from W to Z.

import pandas as pd
import numpy as np
df = pd.DataFrame(np.random.randn(5,4),index='A B C D E'.split(),columns='W X Y Z'.split())

Let’s take a look at another example-

import pandas as pd
data = [['X', 10], ['Y', 15], ['Z', 20]]
df = pd.DataFrame(data, columns = ['Name', 'Age'])

The above example creates two columns with putting first elements from all internal lists from data, under the column Name and then second elements under the column Age.

If we want to create a dataframe by reading a CSV file, we can use the following:

df = pd.read_csv('file_name.csv')

You can use a few other options described below:

  1. If you want to import only a subset of columns, you can define which columns to import by using the option usecols 
     pd.read_csv('file_name.csv', usecols= ['column_name_1','column_name_2'])

    or the indices of the columns

     pd.read_csv('file_name.csv',usecols=[1,2,3])
  2. If your CSV file does not have column headers, set header=None 
     df.read_csv('file_name.csv’, header=None)
  3. If you want to use a particular column as index, use the option index_col 
     pd.read_csv('file_name.csv', index_col='column_name_to_set_index')
  4. Import a range of rows from the file. The following example reads rows $31$ to $45$. skiprows skips the first $30$ while reading first $45$ rows using nrows. 
     df = pd.read_csv('file_name.csv', dtype=float, skiprows=30, nrows=45)

Dataframe Operations

  1. Get a summary of the dataframe using describe() 
     df.describe()
  2. Get unique or non-unique values of a column 
     df['col_name'].unique()

     df['col_name'].nunique()
  3. Access a particular column 
     df['target_column_name']
  4. Accessing multiple columns 
     df[['one_column_name','another_column_name']]
  5. Accessing first $n$ or last $n$ number of rows. head(n) is used to access first $n$ rows and tail(n) is used to access last $n$ rows. 
     df.head(5)

     df.tail(15)
  6. Count number of appearences of a value 
     data['col_name'].value_counts()
  7. Creating a new column 
     df['new_column_name'] = df['column_name_1'] + df['column_name_2']
  8. Deleting a column 
     df.drop('target_column_name',axis=1)
  9. Deleting a row 
     df.drop('row_index',axis=0)
  10. selecting a row using loc or iloc 
    df.loc['row_index']

    df.iloc[row_index]
  11. Accessing a single value 
    df.loc['row_index','column_header']
  12. Accessing values from selected multiple rows and multiple columns 
    df.loc[['row_index_1','row_index_2'],['column_header_1','column_header_2']]
  13. creating a new CSV file 
    df.to_csv('file_name.csv')

    exporting without index

    df.to_csv('file_name.csv', index=False)

    sometimes you can get the UnicodeEncodeError. To avoid that

    df.to_csv('file_name.csv', encoding='utf-8')

    export particular columns only

    dt.to_csv('file_name.csv',columns=['col_1','col_2'])

Dataframe Visualization (Graphs)

pandas module offers some direct visualizations (using matplotlib in the background).

  1. bar plot 
     df.plot.bar()

    or a stacked bar plot

     df.plot.bar(stacked=True)
  2. histogram 
     df['column_name'].hist()
  3. line plot 
     df.plot.line(y='column_name',figsize=(10,3),lw=1)
  4. scatter plot 
     df.plot.scatter(x='column_1',y='column_2')

    using a colormap

     df.plot.scatter(x='col_1',y='col_2',c='col_3',cmap='coolwarm')
  5. box plot 
     df.plot.box()
  6. density plot 
     df.plot.density()

In this post, I just showed a few basic uses of the module pandas. In the next post, we will take a more elaborate look at the module.

For accessing all data science in python related posts, check this post:

Collection of Data Science in Python Posts in my Blog.

Have a nice day, cheers!

Leave a comment

You may also enjoy