Switch branches/tags
Nothing to show
Find file History
emunsing
emunsing Added html; cleaned README.md
No change to python file, but added the referenced html file
Latest commit c2a0ecc Feb 9, 2015

README.md

CAISO LMP Location Scraper

Data returned from OASIS is given in terms of LMP node codes, representing ~5000 locations in the California grid for which CAISO calculates a price for electricity, which ultimately is paid to the generators by the utilities.

The node codes provided by CAISO are mostly indecipherable, but CAISO provides a Google Maps visualization of the current price data, which pipes in geographic node coordinates from a node location xml file. With this location information, it is easier to understand how transmission congestion, weather, and loads dictate electricity prices.

This version was built on an old visualization that was used through December 2014, but appears to no longer be hosted. To stay updated, the scraper needs to be adapted to the new visualizer. The scraper outputs a .json file that can easily be converted to .csv (included). This CSV file was published to a Google Fusion Table for greater accessibility to the public, and the .json file is used by the Electricity Mapper project and its associated github project. Check out Cleanweb Berkeley for more projects!

Files

  • DAM_POINTMap.html : The old visualization included the full set of node information in the html source; the Python file takes this as input
  • PointMapStripper.py : The html file has all of the node information in one text block without any tags to allow for machine reading, so BeautifulSoup seemed more awkward than an old-fashioned text crawler. This script crawls through the downloaded html file to extract the node name, generation type, latitude, and longitude, and save it to an output file (.json format)
  • LMP_Nodes.json : The 2247 nodes for which geographic data is provided, as a .json file for machine reading
  • LMP_Nodes.csv : The same data as a .csv for easy manipulation and human reading
  • LMPLocations_vs_FullList.xls : A comparison of the ~5000 nodes for which OASIS provides LMP data, and the smaller set of ~2250 nodes for which we have geographic data. Aggregated Pricing Nodes (APNodes) are included as well.

To Do's:

Future Developments

License

MIT