Skip to main content

Posts

Showing posts from April, 2015
Beautiful Soup HTML parsing The following Python code fetches the specific windspeed web page and extracts the timestamp, average windspeed, direction, gust speed and writes out data to a date stamped file named say /home/user/wind_data/windspeed_date(2015-04-21-12).txt. Schedule a cron job to run this every day at midnight say. The windspeed file can be selected for a particular day and processed by graph.py.  #!/usr/bin/python import os import requests import time from bs4 import BeautifulSoup date_stamp = time.strftime('%Y-%m-%d-%H',(time.localtime(time.time()))) outfile = os.path.join(os.path.expanduser('~'), 'wind_data', "windspeed_%s.txt"%date_stamp) f = open(outfile,'w') list = [] r = requests.get("http://xxxxx.wwww.yyyyy") soup = BeautifulSoup(r.content) table = soup.find("table", {"id":"grid"}) for line in table.findAll('tr'): for l in line.findAll('td'): ...