Data scraping application with Scrapy
Tran, Long (2023)
Tran, Long
2023
All rights reserved. This publication is copyrighted. You may download, display and print it for Your own personal use. Commercial use is prohibited.
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:amk-2023052915247
https://urn.fi/URN:NBN:fi:amk-2023052915247
Tiivistelmä
Because of the popularity of data nowadays, many businesses and companies are interested in any means of collecting data. The objective of the thesis project was to research and implement a robust solution for gathering and extracting data from any website.
The thesis utilized Scrapy, a Python web scraping framework, to solve the mentioned problem. To study and research the functionality of Scrapy, this project attempted to scrape Amazon, one of the largest e-commerce websites. The paper also briefly introduced other available frameworks for scraping. Moreover, the scraping policy and other related problems were discussed as well.
The project successfully retrieved various pieces of information from Amazon and exported the data into several formats. In addition, this study has been a great help to understand many different elements of Scrapy.
The thesis utilized Scrapy, a Python web scraping framework, to solve the mentioned problem. To study and research the functionality of Scrapy, this project attempted to scrape Amazon, one of the largest e-commerce websites. The paper also briefly introduced other available frameworks for scraping. Moreover, the scraping policy and other related problems were discussed as well.
The project successfully retrieved various pieces of information from Amazon and exported the data into several formats. In addition, this study has been a great help to understand many different elements of Scrapy.