Web Crawler / Data Cleanser

Skills required: mySQL stored programs, Hibernate, Spring, JDBC, threaded programming, SVN Much of working code already exists - this is mostly a refactoring and documentation project. Will work with an experienced Java developer Project Lead / Architect. We are NOT using Lucene. Web Crawler Analyze existing functioning code base, refactor to improve maintainability Implement new features to allow crawler to crawl more sites with minimum of configuration changes Refactor crawl process to handle 1000s of crawling threads with good performance Refactor crawl process to crawl targeted sites several times a day with good performance Document refactored code implementation Document configuration process and crawling process Work with Project Lead to make crawler operational on production servers (hosted remotely) Data Cleanser - Develop 100% new Java code, calling existing mySQL stored programs (SP), refactoring mySQL stored programs, writing new stored programs. Documentation of what code calls which SP, which SP does what etc is 40% of the work. Analyze patterns of crawled data to identify errors and potential errors Analyze existing data cleansing mySQL stored programs (70% existing) Refactor existing stored programs, develop new ones and maintain data cleansing stored programs Develop using Java, mySQL stored programs to create data cleansing process Document developed systems Work with Project Lead to make Data Cleanser operational on production servers (hosted remotely)