Use the new non-tenant rubin-pp-dev* buckets in upload tests

doc/playbook.rst

@@ -71,16 +71,16 @@ Buckets
 
 `This document <https://confluence.lsstcorp.org/display/LSSTOps/USDF+S3+Bucket+Organization>`_ describes the overall organization of S3 buckets and access at USDF.
 
-The bucket ``rubin:rubin-pp`` holds incoming raw images.
+The bucket ``rubin-pp-dev`` holds incoming raw images.
 
-The bucket ``rubin:rubin-pp-users`` holds:
+The bucket ``rubin-pp-dev-users`` holds:
 
-* ``rubin:rubin-pp-users/central_repo/`` contains the central repository described in `DMTN-219`_.
+* ``rubin-pp-dev-users/central_repo/`` contains the central repository described in `DMTN-219`_.
   This repository currently contains a copy of HSC RC2 data, uploaded with ``make_hsc_rc2_export.py`` and ``make_template_export``.
 
-* ``rubin:rubin-pp-users/unobserved/`` contains raw files that the upload script(s) can draw from to create incoming raws.
+* ``rubin-pp-dev-users/unobserved/`` contains raw files that the upload script(s) can draw from to create incoming raws.
 
-``rubin:rubin-pp`` has had notifications configured for it; these publish to a Kafka topic.
+``rubin-pp-dev`` has had notifications configured for it; these publish to a Kafka topic.
 
 The default Rubin users' setup on ``rubin-devl`` includes an AWS credential file at the environment variable ``AWS_SHARED_CREDENTIALS_FILE`` and a default profile without read permission to the prompt processing buckets.
 A separate credential for prompt processing developers is at  `vault <https://vault.slac.stanford.edu/ui/vault/secrets/secret/show/rubin/usdf-prompt-processing-dev/s3-buckets>`_ and can be set up as another credential profile for Butler or command line tools such as AWS Command Line Interface and MinIO Client.
@@ -114,13 +114,12 @@ To inspect buckets with the MinIO Client ``mc`` tool, first set up an alias (e.g
     mc alias set usdf-pp https://s3dfrgw.slac.stanford.edu ACCESS_KEY SECRET_KEY
     mc ls usdf-pp/rubin:rubin-pp
 
-
 For Butler not to complain about the bucket names, set the environment variable ``LSST_DISABLE_BUCKET_VALIDATION=1``.
 
 Central Repo
 ============
 
-The central repo for development use is located at ``s3://rubin:rubin-pp-users/central_repo/``.
+The central repo for development use is located at ``s3://rubin-pp-dev-users/central_repo/``.
 You need developer credentials to access it, as described under `Buckets`_.
 
 Migrating the Repo
@@ -259,7 +258,7 @@ tester
 ``python/tester/upload.py`` and ``python/tester/upload_hsc_rc2.py`` are scripts that simulate the CCS image writer.
 It can be run from ``rubin-devl``, but requires the user to install the ``confluent_kafka`` package in their environment.
 
-You must have a profile set up for the ``rubin:rubin-pp`` bucket (see `Buckets`_, above).
+You must have a profile set up for the ``rubin-pp-dev`` bucket (see `Buckets`_, above).
 
 Install the Prompt Processing code, and set it up before use:
 
@@ -269,7 +268,7 @@ Install the Prompt Processing code, and set it up before use:
     setup -r prompt_processing
 
 The tester scripts send ``next_visit`` events for each detector via Kafka on the ``next-visit-topic`` topic.
-They then upload a batch of files representing the snaps of the visit to the ``rubin:rubin-pp`` S3 bucket, simulating incoming raw images.
+They then upload a batch of files representing the snaps of the visit to the ``rubin-pp-dev`` S3 bucket, simulating incoming raw images.
 
 Eventually a set of parallel processes running on multiple nodes will be needed to upload the images sufficiently rapidly.
 
@@ -282,7 +281,7 @@ Sample command line:
    python upload.py HSC 3
    python upload.py LATISS 3
 
-This script draws images stored in the ``rubin:rubin-pp-users`` bucket.
+This script draws images stored in the ``rubin-pp-dev-users`` bucket.
 
 * For HSC, 4 groups, in total 10 raw files, are curated.
   They are the COSMOS data as curated in `ap_verify_ci_cosmos_pdr2 <Rhttps://github.com/lsst/ap_verify_ci_cosmos_pdr2>`_.

python/tester/upload.py

@@ -130,10 +130,10 @@ def main():
     kafka_url = "https://usdf-rsp-dev.slac.stanford.edu/sasquatch-rest-proxy/topics/test.next-visit"
     endpoint_url = "https://s3dfrgw.slac.stanford.edu"
     s3 = boto3.resource("s3", endpoint_url=endpoint_url)
-    dest_bucket = s3.Bucket("rubin:rubin-pp")
+    dest_bucket = s3.Bucket("rubin-pp-dev")
     dest_bucket.meta.client.meta.events.unregister("before-parameter-build.s3", validate_bucket_name)
 
-    src_bucket = s3.Bucket("rubin:rubin-pp-users")
+    src_bucket = s3.Bucket("rubin-pp-dev-users")
     src_bucket.meta.client.meta.events.unregister("before-parameter-build.s3", validate_bucket_name)
 
     last_group = get_last_group(dest_bucket, instrument, date)

python/tester/upload_hsc_rc2.py

@@ -70,7 +70,7 @@ def _set_s3_bucket():
         global dest_bucket
         endpoint_url = "https://s3dfrgw.slac.stanford.edu"
         s3 = boto3.resource("s3", endpoint_url=endpoint_url)
-        dest_bucket = s3.Bucket("rubin:rubin-pp")
+        dest_bucket = s3.Bucket("rubin-pp-dev")
         dest_bucket.meta.client.meta.events.unregister("before-parameter-build.s3", validate_bucket_name)