Getting 404 errors #3

Open
dportnoy opened this Issue Oct 4, 2016 · 20 comments

Comments

Projects
None yet
7 participants
@24AheadDotCom

This comment has been minimized.

Show comment
Hide comment
@24AheadDotCom

24AheadDotCom Oct 18, 2016

I sent an email to usagov-developers at gsa.gov about this 6 days ago and I haven't heard back. I'm going to tweet them and see if that works.

I sent an email to usagov-developers at gsa.gov about this 6 days ago and I haven't heard back. I'm going to tweet them and see if that works.

@cooperfellows

This comment has been minimized.

Show comment
Hide comment
@cooperfellows

cooperfellows Nov 8, 2016

Seeing the same issue in a project I had been using this data. Did you ever get an update?

Seeing the same issue in a project I had been using this data. Did you ever get an update?

@MayankSingla13

This comment has been minimized.

Show comment
Hide comment
@MayankSingla13

MayankSingla13 Dec 19, 2016

I am getting this error too. I sent an email but haven't heard back. I need list of all agencies and their parent agencies so that I can create a hierarchy for a project.

I am getting this error too. I sent an email but haven't heard back. I need list of all agencies and their parent agencies so that I can create a hierarchy for a project.

@thedore17

This comment has been minimized.

Show comment
Hide comment
@thedore17

thedore17 Feb 5, 2017

Anyone ever get any resolution to this?

Anyone ever get any resolution to this?

@MayankSingla13

This comment has been minimized.

Show comment
Hide comment
@MayankSingla13

MayankSingla13 Feb 5, 2017

No man. I have sent multiple emails but never got a reply.

No man. I have sent multiple emails but never got a reply.

@thedore17

This comment has been minimized.

Show comment
Hide comment
@thedore17

thedore17 Feb 5, 2017

@cooperfellows

This comment has been minimized.

Show comment
Hide comment
@cooperfellows

cooperfellows Feb 5, 2017

@MayankSingla13

This comment has been minimized.

Show comment
Hide comment
@MayankSingla13

MayankSingla13 Feb 5, 2017

I used this: https://www.federalregister.gov/developers/api/v1/
The federal register is old. Some agencies don't exist anymore but they are listed in there. Plus, new agencies have been added in the govt - they are not listed as well.

Hey @cooperfellows can you please share that?

I used this: https://www.federalregister.gov/developers/api/v1/
The federal register is old. Some agencies don't exist anymore but they are listed in there. Plus, new agencies have been added in the govt - they are not listed as well.

Hey @cooperfellows can you please share that?

@thedore17

This comment has been minimized.

Show comment
Hide comment
@thedore17

thedore17 Feb 5, 2017

Hey @cooperfellows -- definitely would be great if you're willing to share that. @MayankSingla13 Thanks for the Federal Register link. It would work for me if it had address or email fields, but it doesn't have either. There is a main website field that I might be able to regex email domains out of, but my god that doesn't sound fun.

What I find really frustrating is that clearly this API still exists and functions because their own website is clearly using it. That's our government for you....

Thanks for your help guys.

Hey @cooperfellows -- definitely would be great if you're willing to share that. @MayankSingla13 Thanks for the Federal Register link. It would work for me if it had address or email fields, but it doesn't have either. There is a main website field that I might be able to regex email domains out of, but my god that doesn't sound fun.

What I find really frustrating is that clearly this API still exists and functions because their own website is clearly using it. That's our government for you....

Thanks for your help guys.

@cooperfellows

This comment has been minimized.

Show comment
Hide comment
@cooperfellows

cooperfellows Feb 7, 2017

Hey @thedore17 attached is a static list from around Mid September of last year. It's not the complete set of data for each agency, but it's a good chunk of it. I want to be clear that this data could be out of date/incorrect, as I found the API to be lacking in accurate data now and then, but its certainly better than nothing.
Federal Agencies via API as of 2016-09-14.xlsx

Hey @thedore17 attached is a static list from around Mid September of last year. It's not the complete set of data for each agency, but it's a good chunk of it. I want to be clear that this data could be out of date/incorrect, as I found the API to be lacking in accurate data now and then, but its certainly better than nothing.
Federal Agencies via API as of 2016-09-14.xlsx

@osnofas

This comment has been minimized.

Show comment
Hide comment
@osnofas

osnofas Feb 7, 2017

@cooperfellows I cannot, just cannot believe the luck that I just came to this place and you posted that file. Made my day. Thanks!

osnofas commented Feb 7, 2017

@cooperfellows I cannot, just cannot believe the luck that I just came to this place and you posted that file. Made my day. Thanks!

@cooperfellows

This comment has been minimized.

Show comment
Hide comment
@cooperfellows

cooperfellows Feb 7, 2017

@osnofas Glad you will find it helpful, but again, I take no responsibility for the accuracy of all that data.

It's what the API returned in mid-September of 2016, and as noted, the API was not always accurate.

@osnofas Glad you will find it helpful, but again, I take no responsibility for the accuracy of all that data.

It's what the API returned in mid-September of 2016, and as noted, the API was not always accurate.

@osnofas

This comment has been minimized.

Show comment
Hide comment
@osnofas

osnofas Feb 7, 2017

I see what you mean - apparently it thinks that Washington is in Arkansas.

osnofas commented Feb 7, 2017

I see what you mean - apparently it thinks that Washington is in Arkansas.

@cooperfellows

This comment has been minimized.

Show comment
Hide comment
@cooperfellows

cooperfellows Feb 7, 2017

Yep, we wound up using "Zip" instead of the "Statter" field because AK (which is Alaska) kept showing up for Agencies in Washington DC. The Zip field tended to be more reliable, when it was available, because not every record had one :), depending on your use case, you could use a geo lookup API to get the correct state based on that Zip.

My guess is that "AK" is the default value for the "Stateter" field in whatever system people used when entering this data into the API (AK is the first state if abbreviations are listed alphabetically)

cooperfellows commented Feb 7, 2017

Yep, we wound up using "Zip" instead of the "Statter" field because AK (which is Alaska) kept showing up for Agencies in Washington DC. The Zip field tended to be more reliable, when it was available, because not every record had one :), depending on your use case, you could use a geo lookup API to get the correct state based on that Zip.

My guess is that "AK" is the default value for the "Stateter" field in whatever system people used when entering this data into the API (AK is the first state if abbreviations are listed alphabetically)

@thedore17

This comment has been minimized.

Show comment
Hide comment
@thedore17

thedore17 Feb 7, 2017

@cooperfellows

This comment has been minimized.

Show comment
Hide comment
@cooperfellows

cooperfellows Feb 7, 2017

I didn't work with the data too much myself, handed it off to someone else on the team for import into another system. The "stateter" and "Fed Branch" columns provided the most trouble for him from what I could dig up. Not sure about the Address field. It wasn't a big enough problem that I heard about it, so I guess that's good?

I didn't work with the data too much myself, handed it off to someone else on the team for import into another system. The "stateter" and "Fed Branch" columns provided the most trouble for him from what I could dig up. Not sure about the Address field. It wasn't a big enough problem that I heard about it, so I guess that's good?

@thedore17

This comment has been minimized.

Show comment
Hide comment
@thedore17

thedore17 Feb 20, 2017

Hey Folks, I went through the doc kindly provided by @cooperfellows to clean up all the errant AK's. Interestingly enough there was only one accurate AK out of the entire 474 office list. I am sharing it as a Google Doc here. I also made this data more hacker friendly by converting it to JSON. Please let me know if you find any issues: https://github.com/thedore17/FederalData/blob/master/FederalAgencies-2016-9-14.json

Hey Folks, I went through the doc kindly provided by @cooperfellows to clean up all the errant AK's. Interestingly enough there was only one accurate AK out of the entire 474 office list. I am sharing it as a Google Doc here. I also made this data more hacker friendly by converting it to JSON. Please let me know if you find any issues: https://github.com/thedore17/FederalData/blob/master/FederalAgencies-2016-9-14.json

@thedore17

This comment has been minimized.

Show comment
Hide comment
@thedore17

thedore17 Feb 20, 2017

Quick update: Combing through the data has yielded other data anomalies and inconsistencies. Many of the address 1 fields are messed up, so I am fixing those to the best of my abilities while maintaining as much of the original data as I can.

Quick update: Combing through the data has yielded other data anomalies and inconsistencies. Many of the address 1 fields are messed up, so I am fixing those to the best of my abilities while maintaining as much of the original data as I can.

@carltoncolter

This comment has been minimized.

Show comment
Hide comment
@carltoncolter

carltoncolter Oct 10, 2017

A year later and I am getting the same errors. Was it taken down or is it experiencing a problem?

A year later and I am getting the same errors. Was it taken down or is it experiencing a problem?

@thedore17

This comment has been minimized.

Show comment
Hide comment
@thedore17

thedore17 Oct 11, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment