LLMs and MIMIC

[aegis301]

Hello,

I'm currently researching possibilities for LLMs working in intensive care. MIMIC and especially MIMIC-Note are of high interest for me in this field. However, I was wondering if there are any official guidelines yet in place for working with these models. I assume that I can't just send MIMIC Data over an API like langchain in order to feed them into a running model like ChatGPT. Locally training and running a model should be within the scope of possibilities however, correct? Data safety and reasonable care are high priorities in our research and we want to make sure we're operating within the given legal guidelines.

Thanks so much in advance for the answer.

[alistairewj]

Thank you for being thoughtful about this!

1. It is clear that sending data over an API is a violation of the DUA.
2. It's also clear that by storing data on Google Cloud Platform, we are to some extent trusting GCP to house the data, albeit within our complete control (we can delete it, and the data are encrypted such that GCP staff cannot read it).

For LLMs, we are carrying on with the principles behind that second point. So, as of now (July 2023), our stance is:

1. Sending data to an external API endpoint (Claude, OpenAI, etc): Not permitted.
2. Deploying a model within your controlled cloud environment and using it (e.g. Azure OpenAI): Permitted.
3. Running an LLM locally on your own hardware/access controlled local hardware: Permitted.

For the 2nd option, you often have to request to omit human review of the data. It's easy to do but it's an extra step. If the reasons are asked, they are two-fold: it's highly sensitive data, and you have not been granted the right to share the data with the humans for review.

In general we try to keep data on HIPAA compliant cloud services, to be safe, but it's not a strict requirement.

[aegis301]

Thank you so much for your answer and your work! We will happily comply with your guidelines and hope others will use this thread as guidepost as well.

[aegis301]

I'm sorry @alistairewj but I have another specific question concerning this: Is the MIMIC Demo dataset available for working with APIs like the chatGPT API or hosted LLMs? We are trying to build something with Pandas AI and so far there are no locally versions of it available for working locally. But of course we want to operate within your regulations! Thanks for working with us on this.

[alistairewj]

You can share the demo dataset with the APIs. It is shared under a different license which permits that use.