permits

[Jegelewicz]

Shouldn't we have an option of "export" for permit type? Also, rather than mash types together (take, possess, transport), I think it might be better to let us allow multiple types. Just an idea!

[campmlc]

We are hoping to be able to revise permits soon with some external funding. So please pass on more suggestions! Mariel

…

On Thu, May 18, 2017 at 3:06 PM, Teresa Mayfield ***@***.***> wrote: Shouldn't we have an option of "export" for permit type? Also, rather than mash types together (take, possess, transport), I think it might be better to let us allow multiple types. Just an idea! — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1139>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AOH0hEbK3fHaZfRhGNGKpFLdzvFhVqj0ks5r7LLBgaJpZM4Nfy8a> .

[Jegelewicz]

OK great! It would be nice to attach a document (PDF of the permit) and instead of remarks, be able to add number of specimens allowed to be taken/transported, give details about where the transport is allowed, and a list of species allowed to be taken/transported under the permit.

[Jegelewicz]

I guess we are going to have to consider renewals and how that affects some of the items I mentioned above. It might make sense to have the data for each renewal.

[dustymc]

This has now been funded, bumping priority.

mash types together

https://arctos.database.museum/info/ctDocumentation.cfm?table=CTPERMIT_TYPE

While normalizing type is certainly "correct" from a data modeling standpoint, I'm a little hesitant to suggest adding that complexity for so few values. Are the current values something like "complete"? (E.g., can anyone see needing dozens/hundreds of new permit types?)

Actually, I'm not sure if the smashing is necessary with our current data. I think we need to define values before anything else. I started a spreadsheet. Anyone can edit; please do.

attach a document

#1165

add number of specimens allowed to be taken/transported

Can that be structured? I think it's usually unpredictable free text: "17 sparrows, 5 warthogs, ..."

where the transport is allowed

Example of those data? (Isn't that usually somewhat embedded in Issuer? AK can issue permits for AK...)

renewals

http://handbook.arctosdb.org/documentation/permits.html#renewed-date

[dustymc]

buy secure permit storage??

Investigate, report to AWG.

[dustymc]

TACC can provide "private" storage starting at $118/TB/yr.

What are our requirements?

• Do we need HIPAA-level security or is something behind a simple password sufficient?
• Do we need individual passwords, or would one shared account work?
• ????

[ccicero]

I edited Dusty's google doc with my suggestions for permit type values and associated definitions. We need a way to attach >1 permit type to a permit.

Good to hear about 'private' storage option at TACC. What does it mean to have HIPAA-level security, how would that work?

Sounds like something that we might want to bounce to the ASC, but can we draft a brief summary of the different options to send to them?

[ccicero]

Also, there may be other types of permits for cultural collections - Angie?

[dustymc]

way to attach >1 permit type to a permit.

OK, I can get started on that. Shouldn't be TOO complicated given how permits relate to other stuff. That model also allows for zero types, although I can "strongly discourage" that in the interfaces. OK?

What does it mean to have HIPAA-level security, how would that work?

https://www.hhs.gov/hipaa/for-professionals/security/laws-regulations/index.html

Looks like end-to-end encryption and server-side compliance monitoring, but I have no idea how that's implemented. Almost certainly incompatible with shared accounts.

TACC does allow individual accounts to the secure storage, but it would be extra work/maintenance for us. I suspect that would get very complicated when staff changes, and I think it would eliminate any chance of sharing a permit (eg, when material is divided among multiple institutions). I think simpler is better, when/if it's possible.

I think we should develop a general idea of what we need and then schedule a call with TACC for details.

@AJLinn

[AJLinn]

The cultural collections, primarily archaeology, would have land use permits for excavations/surveys that are issued by state, federal, and tribal entities (these would fall under the "collect" permit type in the Google doc). These would most likely be associated at the accession level and they should only be visible by operators of the UAM:Arc collection. Is that possible in this proposed system?

When we loan objects made from marine mammal or other protected species across international borders, we use the same permits as the natural history collections (e.g., CITES, MMPA affidavits, NMFS, etc.). Those would be media associated with Loans and would correspond with "export" and "import" types.

The other kind of non-public media we'd want to have associated with our specimens would be documentation relating to NAGPRA work (Native American Graves Protection and Repatriation Act). This could be correspondence between tribes and the museum where we discuss a repatriation claim, which might involve human remains, funerary objects, sacred objects, objects of cultural patrimony. The way we track this at the object level is specimen record-->loan (data)-->project. The media would be associated with the Project. For example: http://arctos.database.museum/project/central-council-of-the-tlingit-and-haida-indian-tribes-nagpra-consultation-documentation

These wouldn't be permits per se, but important media that document our compliance with a federal regulation. These would need to restricted to operators of that collection as well (UAM:EH and UAM:Arc).

Thoughts?

[ccicero]

Maybe we need something about regulatory act governing the permit. e.g., CITES - permit type = import or export, issuing agency = U.S. Fish and Wildlife Service, no clear place to put that it's a CITES permit except remarks. Could we add a field for Regulation that is a controlled list (not required), e.g.

Convention on International Trade in Endangered Species
U.S. Endangered Species Act
Migratory Bird Treaty Act
Marine Mammal Protection Act
Native American Graves Protection and Repatriation Act
etc.

Re: permissions, simpler is probably better but how simple? Sounds like cultural collections wouldn't want others to see their permits for land use? Other collections probably feel the same way, e.g,. would UAM:Bird want MVZ:Bird to have access to copies of its permits (probably not)? Is 'by collection' too complex? Something we need to discuss at next AWG meeting.

[dustymc]

land use permits

@ccicero is that the same as "research" (which still makes no sense to me - do we have specimens which permits DON'T allow to be used for research?!?)

associated at the accession level

All permits tie in to transactions (accessions, loans, borrows)

only be visible by operators of the UAM:Arc collection

Permits are currently shared. Please file an Issue if we need to re-address that (or only enter the information which can be shared across Arctos). No permit information is publicly shared - GGBN will receive summarized data only.

media ... restricted to operators of that collection

All Media data objects (eg, organized text fields) are shared.

All Media files (eg, images and PDFs) hosted by Arctos (eg, on https://web.corral.tacc.utexas.edu/) are publicly available.

Arctos Media URI is unrestricted - you can host your own media behind whatever protections you want (eg, your own FTP/web/whatever server which you administer) and link to that from Media - users will end up on your password page (or wherever you send them).

I'm going to split the discussions regarding Arctos hosting restricted Media off into it's own Issue. #1328

[dustymc]

field for Regulation that is a controlled list

Is that a single value? EG are there CITES+ESA+... permits?

Is 'by collection' too complex?

To administer: probably not.

To use: depends. What happens when CollectionA keeps the skulls and sends tissues to CollectionB?

[ccicero]

I don't think land use is necessarily the same as research. For example, for Guatemala we have a Convenio that is a collaborative research agreement between the MVZ and Universidad de San Carlos. In other cases, there may be a land use agreement that allows you to dig on Forest Service lands. Two separate kinds of permissions.

Re: Regulation: No, those are separate acts and thus have separate permits. One accession or loan might have a CITES permit and a ESA permit, but they are issued as separate permits.

[dustymc]

collaborative research agreement between the MVZ and ...

What ACTION does that allow though??

[ccicero]

In Guatemala we have a Convenio which authorizes us to do research in Guatemala as a collaboration with the University there. It's a 5 year agreement that facilitates us getting permits to collect and export, but does not have specific authorizations. So to do work in Guatemala we have the Convenio, collecting permit with specific authorizations, and export permit. I think it's the same in Indonesia where each person first needs to get a permit that authorizes them to do research in the country, then specimens are covered by collecting and export permits.

There are also 'special-use permits' e.g., from USFS, probably the same as a land use permit.
https://www.fs.fed.us/working-with-us/contracts-commercial-permits/special-use-permit-application

[dustymc]

Got it - thanks.

Here's where I think we are:

Permit is currently...

UAM@ARCTOSTE> desc permit
 Name								   Null?    Type
 ----------------------------------------------------------------- -------- --------------------------------------------
 PERMIT_ID							   NOT NULL NUMBER
 ISSUED_BY_AGENT_ID						   NOT NULL NUMBER
 ISSUED_DATE								    DATE
 ISSUED_TO_AGENT_ID						   NOT NULL NUMBER
 RENEWED_DATE								    DATE
 EXP_DATE								    DATE
 PERMIT_NUM								    VARCHAR2(25)
 PERMIT_TYPE							   NOT NULL VARCHAR2(50)
 PERMIT_REMARKS 							    VARCHAR2(300)
 CONTACT_AGENT_ID							    NUMBER

and will become...

UAM@ARCTOSTE> desc permit
 Name								   Null?    Type
 ----------------------------------------------------------------- -------- --------------------------------------------
 PERMIT_ID							   NOT NULL NUMBER
 ISSUED_BY_AGENT_ID						   NOT NULL NUMBER
 ISSUED_DATE								    DATE
 ISSUED_TO_AGENT_ID						   NOT NULL NUMBER
 RENEWED_DATE								    DATE
 EXP_DATE								    DATE
 PERMIT_NUM								    VARCHAR2(25)
 PERMIT_REMARKS 							    VARCHAR2(300)
 CONTACT_AGENT_ID							    NUMBER
requiring_act NEW FIELD NULLABLE--->new code table with values from ^^ up there

plus...

new table permit_type
---------------------------------------------------
whatverwecallit PKEY
permit_id FKEY-->permit
PERMIT_TYPE-->fkey ctpermit_type			   NOT NULL VARCHAR2(50)

Are we all happy with everything else? Should we break Agents out into a table (agent+role) like we've done with transactions, or ?????

Things NOT in this, because I have no idea what to do with them:

add number of specimens allowed to be taken/transported
where the transport is allowed

[campmlc]

Can we allow more than one permit contact?

…

On Nov 14, 2017 3:00 PM, "dustymc" ***@***.***> wrote: Got it - thanks. Here's where I think we are: Permit is currently... ***@***.***> desc permit Name Null? Type ----------------------------------------------------------------- -------- -------------------------------------------- PERMIT_ID NOT NULL NUMBER ISSUED_BY_AGENT_ID NOT NULL NUMBER ISSUED_DATE DATE ISSUED_TO_AGENT_ID NOT NULL NUMBER RENEWED_DATE DATE EXP_DATE DATE PERMIT_NUM VARCHAR2(25) PERMIT_TYPE NOT NULL VARCHAR2(50) PERMIT_REMARKS VARCHAR2(300) CONTACT_AGENT_ID NUMBER and will become... ***@***.***> desc permit Name Null? Type ----------------------------------------------------------------- -------- -------------------------------------------- PERMIT_ID NOT NULL NUMBER ISSUED_BY_AGENT_ID NOT NULL NUMBER ISSUED_DATE DATE ISSUED_TO_AGENT_ID NOT NULL NUMBER RENEWED_DATE DATE EXP_DATE DATE PERMIT_NUM VARCHAR2(25) PERMIT_REMARKS VARCHAR2(300) CONTACT_AGENT_ID NUMBER requiring_act NEW FIELD NULLABLE--->new code table with values from ^^ up there plus... new table permit_type --------------------------------------------------- whatverwecallit PKEY permit_id FKEY-->permit PERMIT_TYPE-->fkey ctpermit_type NOT NULL VARCHAR2(50) Are we all happy with everything else? Should we break Agents out into a table (agent+role) like we've done with transactions, or ????? Things NOT in this, because I have no idea what to do with them: add number of specimens allowed to be taken/transported where the transport is allowed — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1139 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AOH0hKcqXiYQmAaUpDo_WEwpb7aGhaboks5s2g13gaJpZM4Nfy8a> .

[dustymc]

Can we allow more than one permit contact?

We can if we

break Agents out into a table (agent+role) like we've done with transactions

If we're going there I think we should probably remove all agents from the permit table - having some roles structurally different would just be a confusing mess. So that would be one more table:

permit_agent
-------------------------------
localkey PKEY
permit_id FKEY-->permit
agent_id FKEY-->agent
agent_role FKEY-->new code table

Note that while I can "strongly encourage" agents to exist in interfaces, that structure cannot require agents.

[ccicero]

I think it makes sense to keep one permit contact as the primary contact for notifications, but be able to have >1 'issued to' agent. For example, our USFWS migratory bird permit is issued to me and Rauri, but I am the primary contact who deals with renewals.

I've also been confused by 'renewed_date' versus 'issued_date.' If a permit expires and is renewed, then a new permit would be created with a new issue date and updated expiration date. Can we be clear about when to use 'renewed' versus 'issued' ???

[campmlc]

Separating out agents to a table with agent roles would be useful. I need to be a permit contact and receive notifications for permits obtained by other agents and divisions who have their own contacts.

…

On Tue, Nov 14, 2017 at 3:42 PM, dustymc ***@***.***> wrote: Can we allow more than one permit contact? We can if we break Agents out into a table (agent+role) like we've done with transactions If we're going there I think we should probably remove all agents from the permit table - having some roles structurally different would just be a confusing mess. So that would be one more table: permit_agent ------------------------------- localkey PKEY permit_id FKEY-->permit agent_id FKEY-->agent agent_role FKEY-->new code table Note that while I can "strongly encourage" agents to exist in interfaces, that structure cannot require agents. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1139 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AOH0hBzlHQ9mNzAaxcUNpeJ1oFAUoiMcks5s2hdFgaJpZM4Nfy8a> .

[ccicero]

agreed, it would be better

[dustymc]

AWG Meeting:

• get rid of renewed_date: yes
• forms will "require" "issued to" (but model cannot)
• forms will "require" "issued by" (but model cannot)
• forms will "require" permit type (but model cannot)
• permit type is uncontrolled string - "no number" is a valid value
• change label of "permit number" to "permit identifier"
• Add permit_name (NULL) varchar uncontrolled

• make sure these agent roles appear in agent info

[campmlc]

AWG:

-PERMIT_NUM
-PERMIT_IDENTIFIER (so it reflects different types of identifying systems), required
-Required fields: issued date, exp date, permit identifier, permit type, agent issued by, issued to

[dustymc]

I need a migration pathway for permit_type - from http://arctos.database.museum/info/ctDocumentation.cfm?table=CTPERMIT_TYPE to https://docs.google.com/spreadsheets/d/1UIUxdoS96tdcZMH-dRCKl48T1QKCOktLj5kHtUIxGx8/edit#gid=0

other --> no permit types? Does a 'nothing very useful to say' option mean we DO NOT want to "require" permit type after all??
import --> no change
take/possess -->collect
research --> no change
take/possess, research -->collect + research
take/possess, transport -->collect + transport
transfer of property --> ??? Import?? Export?? Transport??? NULL??
transport --> no change

If I don't hear back soon-ish I'll drop type for the mystery permits and leave remarks.

@ccicero I think the values I need help with are mostly MVZ permits

@jtgiermakowski can you clarify the need for permit_name

[ccicero]

I'm looking at the list of permits that are 'transfer of property' and think that we need another permit type for those. Most of them are USFWS confiscations where they transfer property to the institution once they have done their legal work. I don't see that fitting under any of the other permit types. Basically, the permit authorizes the institution/collection to possess the specimens.

[ccicero]

one option might be to combine 'transport' and 'transfer of property' into a new permit type 'transfer'. The USDA transport permit is really a transfer permit. In looking at the 'transport' permits, many of those are actually export permits. There are a lot of those. Can you put transport and transfer of property permits into a google doc and we can go through and provide a migration path for each one based on what's noted in remarks?

[dustymc]

https://docs.google.com/spreadsheets/d/1XnZ4Jt3SViLYkvDGSrF8A45I7mbMTEfLnBWKpuiBTuY/edit?usp=sharing

create table temp_pmt_tfrpt as
select
  permit_id,
  getPreferredAgentName(ISSUED_BY_AGENT_ID) ISSUED_BY_AGENT,
  getPreferredAgentName(ISSUED_TO_AGENT_ID) ISSUED_TO_AGENT,
  getPreferredAgentName(CONTACT_AGENT_ID) CONTACT_AGENT,
  ISSUED_DATE,
  RENEWED_DATE,
  EXP_DATE,
  PERMIT_NUM,
  PERMIT_TYPE,
  PERMIT_REMARKS
from
  permit
where
  permit_type in ('transport','take/possess, transport','transfer of property')
;

[ccicero]

see updated google doc for definitions - changed 'transport' to 'transfer' and updated the definition.

[ccicero]

I went through the google doc for transport + transfer of property and did my best guess as to new permit type for migration. Cells in orange are uncertain. Most are NMFS - not sure if those cover collect, import, and/or export. Also unclear if permit to Ted Papenfuss for expedition dated 11 Aug 2007 (subpermit to T.J. Papenfuss under Dr. Gabriela Parra Olea for collecting Plethodontid salamanders) is collect + export. @atrox10

[ccicero]

It would be good if someone else reviewed this to see if they concur with my proposed new types for these transport + transfer of property permits.

[dustymc]

Thanks!! I'm going to have code ready for prod and will need to do something fairly soon (early/mid next week??). If there's no resolution by then or you're not comfortable with something or you just want the old data, I can stuff current permit type into remarks and we can revisit this later. Better to have it all cleaned up of course, but not the end of the world if that doesn't happen.

[AJLinn]

I just added seven more regulations, 2 for wildlife and 5 for cultural and paleontological collections. The rest looks good to me.

[dustymc]

Here's the current intended permit type migration path.

Current values and definitions for permit type are:

• collect: Authorization to collect and possess specimens or their parts (e.g., blood, feather samples). NOTE: Same as "take/possess"
• export: Authorization to export specimens or their parts from one country to another.
• import: Authorization to import specimens or their parts from one country to another.
• research: Authorization to conduct research within a permitted jurisdiction. Usually also requires additional permits (e.g., collect, export). Examples: U.S. Forest Service research permit; Memorandum of Understanding; Convenio.
• salvage: Authorization to pick up and possess dead animals.
• transfer: Authorization to transfer specimens from one institution to another within the same country. Example: USDA Transport Permit for restricted materials between US institutions; USFWS authorization to transfer confiscated materials.

And the migration path:

• Where permit_id is in https://docs.google.com/spreadsheets/d/1XnZ4Jt3SViLYkvDGSrF8A45I7mbMTEfLnBWKpuiBTuY/edit#gid=2122230195,
  and permit_type_should_be contains ONLY a "+"-separated list of permits, use permit_type_should_be values (Or should I ignore "?" and use what's left?? @ccicero )

for whatever's left:

• other --> DELETE, append to permit_remark 'original permit type given as "other"'
• import -->import
• take/possess-->collect
• research-->research
• take/possess, research-->collect + research
• take/possess, transport-->collect + transfer
• transfer of property-->transfer
• transport-->transfer

The main permit forms in TEST should be essentially functional at this point. Have a look around, let me know if anything is wonky.

I think the "permit_regulation" values (https://docs.google.com/spreadsheets/d/1UIUxdoS96tdcZMH-dRCKl48T1QKCOktLj5kHtUIxGx8/edit#gid=0) are overly cryptic in the forms.

"Convention on International Trade in Endangered Species of Wild Fauna and Flora" seems overly verbose as well (and everybody knows what CITES means anyway). I have no suggestions, but this looks potentially problematic.

[AJLinn]

I think the "permit_regulation" values (https://docs.google.com/spreadsheets/d/1UIUxdoS96tdcZMH-dRCKl48T1QKCOktLj5kHtUIxGx8/edit#gid=0) are overly cryptic in the forms.

screen shot 2017-12-07 at 10 15 43 am

"Convention on International Trade in Endangered Species of Wild Fauna and Flora" seems overly verbose as well (and everybody knows what CITES means anyway). I have no suggestions, but this looks potentially problematic.

Agreed - some acronyms are pretty easy to understand (most everyone will know MMPA or CITES) but I'm cool with the full name being listed on the dropdown list.

[ccicero]

I think that as long as there is a 'help' link from 'Regulation' that shows the full name of the regulation, it will be ok.

@dustymc: migration path looks ok to me except for 'take/possess, transport' - use the suggested permit type in https://goo.gl/DSHzRo. Should not be 'collect + transfer' as noted above. Some state permits are really just collecting permits that also allow disposition in an institution in another state, so they were listed as 'take/possess, transport' but are just 'collect.' Other such permits are 'collect + export' - depends on the permit, so best to go with values in the spreadsheet.

Keep remarks for all permits.

I emailed Link and Ted about the ones that I had as "?"

[dustymc]

@ccicero I will do what's in https://goo.gl/DSHzRo first and THEN the "for whatever's left" steps to anything not in the spreadsheet. The spreadsheet will take precedence. If all "take/possess, transport" (or whatever) permits are covered in the spreadsheet, then nothing else will happen.

I can APPEND old permit_type (and/or whatever) to permit_remarks. I won't remove any existing remarks (or anything else).

There is a link to the regulation code table (http://arctos-test.tacc.utexas.edu/info/ctDocumentation.cfm?table=CTPERMIT_REGULATION) on the new edit form.

[ccicero]

Can you please put the 'other' permits in that spreadsheet (or another one, whatever's easiest) and I'll go through those as well. They should not be deleted.

Looking at the test form - has that not been updated to the new mockup?

[dustymc]

"other" permits are https://docs.google.com/spreadsheets/d/1ApeqKmiLsagYLmgNbXdUE_kI5wqhAE5GtP-eT39W8gA/edit?usp=sharing

Yes test forms are updated - it's not quite the mockup but the new functionality is there (http://arctos-test.tacc.utexas.edu/Permit.cfm) - what are you seeing? The DATA does not follow the last few changes.

[dustymc]

deleted

I mean delete the TYPE, not the permit. That would look like http://arctos-test.tacc.utexas.edu/Permit.cfm?permit_id=10000511&action=editPermit - you won't be able to save without selecting a type.

[ccicero]

Will look at the 'other' later - can't do it now (xmas party!)

Unclear how you choose a regulation in the new form when creating a new permit?

Is date required? Issued_date at least should be required, not sure about Expiration_date since some 'permits' are documents that don't have a clear end date. ???

[dustymc]

regulation

Create and edit to add more types and regulations.

Fill in the absolute basics, save to get all the options.

date required

Nope.

UAM@ARCTOS> desc permit
 Name								   Null?    Type
 ----------------------------------------------------------------- -------- --------------------------------------------
 PERMIT_ID							   NOT NULL NUMBER
 ISSUED_BY_AGENT_ID						   NOT NULL NUMBER
 ISSUED_DATE								    DATE
 ISSUED_TO_AGENT_ID						   NOT NULL NUMBER
 RENEWED_DATE								    DATE
 EXP_DATE								    DATE
 PERMIT_NUM								    VARCHAR2(25)
 PERMIT_TYPE							   NOT NULL VARCHAR2(50)
 PERMIT_REMARKS 							    VARCHAR2(300)
 CONTACT_AGENT_ID							    NUMBER

Requiring issued date seems reasonable to me, but we'd have to do something about these.

UAM@ARCTOS> select count(*) from permit where issued_date is null;

  COUNT(*)
----------
	52

1 row selected.

[dustymc]

Unless someone stops me in the very near future, I'm going to add another option for permit_type (to make GGBN reporting unambiguous, among other things).

permit not required
definition: Used for transactions under which a permit is explicitly not required. 

I see no other way to make the distinction between "Permit not required" and "Permit not available" at https://terms.tdwg.org/wiki/ggbn:permitStatus.

[dustymc]

This is now in production.

[campmlc]

I just attempted to add a USFWS 3-177 export permit to a loan for the first time since we have updated the permit structure. I am still learning this process, but it seems still to be very unclear and I made a lot of mistakes. I would like to be able to create the permit from the loan page, which I don't think is possible? I see that I can add media directly to the loan, but in this case I wanted to create the permit and add the media to the permit. But when I do so, nothing shows up on the permit page - unless I go to add/edit media, where I can see the it in the box. Shouldn't this be visible from the edit permit screen and also show up in Edit loan?
https://arctos.database.museum/Permit.cfm?Action=editPermit&permit_id=10001438

Also, there seems to be a big lag that occurs after loading media or deleting media - is this normal?

Anyone else tried this process yet, or have suggestions on best way to go about it?