minio manage using mc in console

in portainer, log onto console for s3

configure login

mc alias set local http://localhost:9000 ACCESS SECRET

check:

mc alias ls

how may files in a summoned directory

mc ls --summarize local/gleaner-wf/summoned/ssdbiodp

remove older than 30 days: (remove --dry-run once tested)

mc rm local/dvtest/summoned/geocodes_demo_datasets --older-than 30d -r --dangerous --dry-run --force

clean out the summoned and milled (remove --dry-run once tested)

mc rm local/dvtest/summoned/ local/dvtest/milled/ local/dvtest/prov/ --older-than 30d -r --dangerous --dry-run --force

find by metadata

https://min.io/docs/minio/linux/reference/minio-mc/mc-find.html#find-objects-with-matching-metadata

mc find --metadata="X-Amz-Meta-Url=https://raw.githubusercontent.com/earthcube/GeoCODES-Metadata/main/metadata/Dataset/allgood/argo-20220707.json" local/dvtest/summoned/

lifecycle:

You used to be able to do rules in the minioconsole.

~ % mc ilm rule add geocodes-aws/decoder/scheduler/logs --expiry-days 90
~ % mc ilm rule ls geocodes-aws/decoder
┌──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Expiration for latest version (Expiration)                                                                           │
├──────────────────────┬─────────┬───────────────────────────────────────┬──────┬────────────────┬─────────────────────┤
│ ID                   │ STATUS  │ PREFIX                                │ TAGS │ DAYS TO EXPIRE │ EXPIRE DELETEMARKER │
├──────────────────────┼─────────┼───────────────────────────────────────┼──────┼────────────────┼─────────────────────┤
│ csv2tqjok4ts73a9dm7g │ Enabled │ scheduler/logs                        │ -    │             90 │ false               │
│ csv2tv3ok4ts73a9dm80 │ Enabled │ graphs/archive                        │ -    │            180 │ false               │
│ csv2ucrok4ts73a9dm8g │ Enabled │ milled                                │ -    │             14 │ false               │
│ csv33vbok4ts73a9dm90 │ Enabled │ decoder/reports/*/bucketutil_urls.csv │ -    │             90 │ false               │
└──────────────────────┴─────────┴───────────────────────────────────────┴──────┴────────────────┴─────────────────────┘

% mc ilm rule export geocodes-aws/decoder

{
 "Rules": [
  {
   "Expiration": {
    "Days": 90
   },
   "ID": "csv2tqjok4ts73a9dm7g",
   "Filter": {
    "Prefix": "scheduler/logs"
   },
   "Status": "Enabled"
  },
  {
   "Expiration": {
    "Days": 180
   },
   "ID": "csv2tv3ok4ts73a9dm80",
   "Filter": {
    "Prefix": "graphs/archive"
   },
   "Status": "Enabled"
  },
  {
   "Expiration": {
    "Days": 14
   },
   "ID": "csv2ucrok4ts73a9dm8g",
   "Filter": {
    "Prefix": "milled"
   },
   "Status": "Enabled"
  },
  {
   "Expiration": {
    "Days": 90
   },
   "ID": "csv33vbok4ts73a9dm90",
   "Filter": {
    "Prefix": "decoder/reports/*/bucketutil_urls.csv"
   },
   "Status": "Enabled"
  }
 ]
}