I'm trying to log in to [login to view URL], navigate to a chart url, and then scrape a live price and calculated values from the page. You have to be logged in to get the values.
I would like someone to write a python program that can actively scrape values from the page and write those values to an updateable csv file. Whenever I run the script, I want it to scrape values, the next time I run the script, I want it to scrape new values but keep the old values in the file too.
Program operations would look something like this:
#1. Write one function to open a page with selenium chrome driver and/or Beautifulsoup.
#2. Another to login and navigate to a page ([login to view URL])
#3. Run another function to scrape multiple values off of that chrome/chromedriver page
It's easy enough to do #1 and #2, but scraping the values on #3 is proving difficult to me.
I have either an "xpath" or "div class" to work with, or we can find another way:
xpath: /html/body/div[2]/div[1]/div[3]/div[1]/div/table/tr[1]/td[2]/div/div[2]/div/div[2]/div/div[5]/div[2]
div class: <div class="valueValue-3kA0oJs5" style="color: rgb(33, 150, 243);">6566.97</div>
I thought about doing this all with Beautifulsoup, but I'm not sure how to really get passed the login. Also, I kinda like using selenium chromedriver because it shows me the charts that the program is scraping.
Hi, There
Python web scrapping expert, who have rich experience in beautifulsoup is here.
Your requirements were checked and this job can be completed in a few hours.
Hope to hear from you soon.
Regards.