Python Data Extraction and compare tool
€250-500 EUR
Pagado a la entrega
Goal objective: Create a crawler that checks a set of urls (100-30.000 per client) on a daily basis and
Extracts relevant pre-defined data (H1, Title, Meta-Data, status codes, canonical, as well as optional attributes via xpath, which a user can configure via a front end).
The first time a newly create job is run, the crawled data is saved to a database. From then on each day the newly crawled data is compared against the existing data. If changes are detected (Example H1 changed) an Email is sent with infos to the respective urls and which data changed.
I was thinking about a simple flask web interface with flask admin through which the Urls for each client as well as custom xpath and email can be configured (option to upload xlsx or csv, as well as copy paste, clients deleted, as well as crawl data reviewed (maybe via. datatables)
The whole project should be bundled in a docker compose with all necessary dependancies (Redis, Database, …)
The plan is to integrate further tools after this initial project so this should be also taken into account for but shouldn’t be a problem with a framework such as flask. In case you have further suggestions or a change of tools please feel free to comment.
Nº del proyecto: #27583909
Sobre el proyecto
15 freelancers están ofertando un promedio de €388 por este trabajo
Hi, Greetings! ✅checked your project details: Python Data Extraction and compare tool ✅Completed Time: In project deadline We have worked on 650 + Projects. I have 6 + years of the experience in same kind of pro Más
Hi, I am professional Python script developer, I can build a script for you by python within very short time. I have good experience in Selenium, Flask, Beautiful-soup and proxy rotation. I am scraping expert Please s Más
Hi, I am Data Engineer ,python programmer, and B2B Lead Generation Specialist. have 5 year of professional working Experience in website scraping, data mining, field and Lead Generations. I have already scraped Thousan Más
I'm a Flask expert with rich experience in web development. I read the job details extremely carefully. I think I can complete your project on time and within your budget. I'm ready to do it now. Should you require Más
Hello, l have read your requirements and I can build web app using flask. I am also experienced in web scraping. My programming skills : python, selenium, beautifulsoup, scrapy, lxml, requests, numpy, panda Más
Hello dear! How are you ? I've 5 years experienced in developing field. And I checked your task description and very interested in this project. Dear Client, I am expertise in php, core php, CMS, HTML5, CSS3, bootstrap Más
Hi there, I will complete a python extraction and will provide you with clear, concise notes. I am a PhD writer with 5 years of experience. I have worked on several similar projects , and can deliver quality notes to Más
Hello! I can provide you full support for this project. I have high experience with web technologies and I am pretty sure I can help you. Feel free to contact to provide me more information!
