Build 2 web scraping tools and populate MySQL database!

Stagecoach Software Pty Ltd requires a freelancer to create two automated web scraping tools and a MySQL database suitable to serve as a back end to a forthcoming website to be built on Ruby on Rails. The first tool needs to run on an automated schedule, initially at a set time every week. The tool is required to scrape a single large web site listing thousands of unique products and then populate a MySQL database with details of the products and prices sought. The details to be scraped appear in a consistent format on the site. The first scrape is required to obtain details for all products whilst subsequent scrapes are required to track new products listed or changes of price on existing products. The freelancer must populate the DB using the scraper as part of the project. The second tool is required to automatically download and scrape 2 to 4 pdf consistently formatted text files published every week on a different website that list the realized prices for a significant proportion of the products listed in the first website. The second tool is required to populate a further table in the MySQL database with these products and price results. The MySQL database must then automatically combine the data from the two scraping tools and calculate the difference between the realized price and the initial listed price(s). The DB must be loaded to a host site nominated by me. The scraper tools must be able to be easily activated from the host site and full operating instructions must be provided. Full details of the sites to be scraped and the required DB columns will be supplied to the successful bidder. This project will be followed with the creation of a RoR website that will present graphs showing aggregate and individual data obtained from the database and allow user searches. Preference will be given to bids that make that better facilitate this second stage.