Apple OSX Documentation #86
Conversation
on the orginal searchcode.com one which was uploaded into UNCLEAN. At this point have set the structure up as required and have run parse.py though pep8 ignoring just the line too long errors for the moment. - WIP fetch.sh need to verify the wget settings to avoid overtaxing apple servers and potentially getting banned - WIP parse.py need to modify to work with the new download and ensure that all the links are correct when parsing.
use the folder created by fetch, to resolve the URL's correctly and to parse things out as expected. Strictly speaking this is not "pythonic" but it does the job.
(seriously can Python 3 just work now please). Modifying parse.py to ignore get redirects since we dont need those files. Adding output.txt to the ignore file. Adding redirect to the parse.sh file.
|
@boyter Apologies for the late reply. Checking it out now! |
|
@boyter Ah, does it get the current docs? We updated the Fathead for the current one--I should've made a branch in the Fathead repo instead. Here's an example https://duckduckgo.com/?q=AVAssetReaderVideoCompositionOutput and here's the code https://gist.github.com/jagtalon/78e73bc4f17651684355. Would you be interested in updating your code to get the 10.10 docs instead? https://developer.apple.com/osx/whats-new/ |
|
Easy enough just need to flip the wget to the following, https://developer.apple.com/library/prerelease/mac/sitemap.php and it will pull down the pre-release stuff. I didn't do this because it should change and when it goes live the URL's will change as the current ones are still 10.9. Not sure if apple has a history of trashing the prerelease documentation but there is the risk of getting nothing but 404's. Not sure... ill let you take the call, and if you want the pre release I can flip it over in a second or so. Otherwise the code should remain the same assuming that Apple dosn't change the format in some drastic way between now and release date. |
|
@boyter Alright, well let's work on this current one then and then just update it again when Apple releases the actual docs. :) |
|
Closing for now, would love to get this live though! |
|
@boyter you weren't kidding! I left it overnight and it's still going: https://v.usetapes.com/Ad71d8XFzv |
Updated the OSX documentation fathead parser based on the original one in UNCLEAN.
This matches most of the fathead requirements (I believe) and now pulls down and processes
the documentation from developer.apple.com.
Be aware it takes a very long time to make a mirror of the site using wget. There is a sitemap
provided by Apple (its the start point of the wget command in fetch.sh) which might be able to
speed this up as each link appears to be the correct one.
This still isn't pep8 compliant due to some lines being too long, and isn't 100% Pythonic but
is close enough for the moment.
Finally the output.txt is not added (nor the wget output) as it is well over the 1 meg limit specified.