基于python爬取母婴用品评论的数据分析 毕业论文+开题报告+源码及数据分析
摘 要近年来，随着人们生活品质的不断提高，消费观念亦在不断升级。自2016年国家开放二胎政策以来，母婴市场愈来愈繁荣，母婴商品种类也在不断地增多，人们在进行电商浏览母婴用品时，常常是通过店铺销量和已购买用户的评论进行选购的，面对如此繁多地商品以及网络购物中的不确定性，对大量的评论进行整合和处理，帮助消费者进行客观的判断从而选购到合适的母婴用品是十分有必要的。对于大量评论的热销母婴商品，消费者如果靠阅读评论得到某母婴商品全面性的分析，工作量极大，耗时长，获取的信息也有限。网络爬虫作为一个自动提取网页的程序,可以高效地从数据库的海量信息中挖掘出有效的信息，大量数据可以得到处理和分析。而爬虫系统可以不断抓取网页数据到本地，经过一系列的分析、过滤，从而得到高质量的数据。本文主要从用户ID、评论内容、会员级别、点赞数、回复数、评价星级、购买时间、母婴品牌多维度进行分析，收集官网提供的数据集与Python爬取的日志信息，利用MySQL对数据进行预处理与存储，使用python进行分析与可视化呈现，最终对分析结果进行说明并提供相应意见。关键词：Python；MySQL；电商平台；母婴商品；数据分析AbstractIn recent years, with the continuous improvement of people's quality of life, the consumption concept is also constantly upgrading.Since 2016 national policy open 2 children, maternal and baby market, more and more products are increasing, people in electricity browsing maternal and baby products, often through store sales and have the comments to buy users to choose and buy, in the face of such a variety of goods and in the uncertainty in the network shopping, a large number of comments integration and processing, help consumers to make objective judgment to choose and buy to the appropriate mother and baby products is very necessary.For a large number of hot-selling maternal and baby products, if consumers get a comprehensive analysis of a mother and baby product by reading reviews, it is a huge workload, time-consuming and limited information.As a program for automatically extracting web pages, the web crawler can efficiently excavate effective information from the large amount of information in the database, and a large amount of data can be processed and analyzed.The crawler system can constantly grab the web page data to the local, after a series of analysis, filtering, so as to get high-quality data.This article mainly from the user ID, comment content, member level, thumb up, reply number, star rating, purchase time, maternal and infant brand multidimensional analysis, collect the data set and Python crawl log information provided by the official website, using MySQL, analyze and visualization using python, and finally explain the analysis results and provide corresponding opinions.Key words: Python; MySQL; e-commerce platform; maternal and child products; data analysis目录摘 要 IAbstract II1 绪论 41.1 背景及意义 41.1.1 背景 41.1.2 目标及意义 41.2 应用现状 41.3 数据模型及主要板块 51.3.1数据模型建立 51.3.2 主要板块 51.4 主要方法及技术路线 61.4.1 主要方法 61.4.2技术路线 62开发环境及技术 72.1硬件设备 72.1.1 CPU及操作系统 72.1.2 其他 72.2软件及IDE 72.2.1 MySQL 72.2.2 IDE 82.3主要技术 82.3.1 MySQL 82.3.2 Python 93 可行性分析与模块设计 103.1 可行性分析 103.1.1 技术可行性分析 103.1.2 数据可获得性分析 103.2 各模块设计 103.2.1 数据获取方法 103.2.2 数据预处理设计 103.2.3 数据分析思路设计 124 系统实现与结论 134.1数据采集 134.1.1确定待采集数据 134.1.2确定采集对象及采集思路 134.2采集准备 144.3开始采集数据 144.3.1分析网页 144.3.2思考 194.4数据清洗与分析 204.4.1清洗过程简述 204.5 可视化与分析 204.5.1消费时段分析 214.5.2月消费与会员等级分析 234.5.3评论内容分析 255 设计过程中存在的问题和解决过程 275.1问题1描述 275.2解决办法 275.3问题2描述 275.4解决办法 286 总结 31致 谢 32参考文献 33
