Python Web Scraping to CSV

Advertisement

Comma Separated Values or (CSV) is one of the most common formats of spreadsheet data storage files. This file extension is very popular in machine learning. Data used in machine learning are stored in CSV’s as working with CSV files in Python is easy with Pandas. CSV is also supported by Microsoft Excel, this is also a huge factor for the popularity of CSV file format. Today in this blog with Hack The Developer, you will learn How you can store Python Web Scraping Data to CSV file.

Writing to CSV File in Python

 

First of all, we need to take a look at How to work with CSV Files in Python, in order to store our Python Web Scraping Data into it.

In this Example Section, where we will learn How to work with CSV in Python. We will save a CSV file at our workspace called test.csv that will contain three Columns and a total of 11 rows. Three Columns will be SR(Serial Number), ID(between 1-100), and Price(between 100-1000).

Here is the Python Code to Save a CSV File:

import csv
import random

csvFile = open('test.csv', 'w+')
try:
    writer = csv.writer(csvFile)
    writer.writerow(('SR', 'ID', 'Price'))
    for i in range(10):
        writer.writerow((i+1, random.randint(1, 100),
                         random.randint(100, 1000)))
finally:
    csvFile.close()

The output of the Above Code. (test.csv)

SR,ID,Price

1,52,582

2,56,717

3,21,648

4,96,305

5,22,349

6,12,334

7,86,918

8,67,369

9,13,497

10,9,425

One thing to keep in mind while working with file handling in Python, if the file i.e test.csv doesn’t already exist, it will be automatically created by Python but if the file already exists, Python will overwrite the file with the new data.

Saving Python Web Scraping to CSV

 

Now, when we have the knowledge about CSV and How to Work with CSV files and Save it in Python. We can move forward to Scrape the Data from the web and save it to our CSV File.

In this example, we are going to scrape Wikipedia’s Comparison of Text Editors.

Here is the code for the Python Web Scraping To CSV:

import csv
from urllib.request import urlopen
from bs4 import BeautifulSoup

html = urlopen(
    'https://en.wikipedia.org/wiki/Comparison_of_text_editors')
bs = BeautifulSoup(html, 'html.parser')

table = bs.findAll('table', {'class': 'wikitable'})[0]
rows = table.findAll('tr')

csvFile = open('Text-Editor-Data.csv', 'wt+')
writer = csv.writer(csvFile)
try:
    for row in rows:
        csvRow = []
        for cell in row.findAll(['td', 'th']):
            csvRow.append(cell.get_text())
            writer.writerow(csvRow)
finally:
    csvFile.close()

This will Save the First Row of the table into our CSV file i.e Text-Editor-Data.csv.

Hope You Like it! Share Your Experience with us.

For More Such Content Stay Tuned to Hack The Developer.

About Author
5 1 vote
Article Rating
Subscribe
Notify of
guest
0 Comments
Inline Feedbacks
View all comments
Scroll to Top