NOAA Upload Script

[BryonLewis]

This is a script for NOAA to import their mirrored bucket annotations into their system. I'm going to deliver it directly to them but I feel it should be included in the repo as a reference point for others who may want to utilize similar features of girder-client.

• Allows specifying a GirderId for a root folder and local mounted folder location to mirror. It will then upload the annotations to the mirrored directory structure in the system.
• ImageSet Structure is :
  • ./bucket-name-images/platform/date/ID/imageType/images/Dir0001
  • ./bucket-name-images/platform/date/ID/imageType/images/Dir0002
  • ./bucket-name-images/platform/date/ID/imageType/images/Dir0003
• Annotation Struture is:
  • ./bucket-name-annotations/platform/date/ID/imageType/images/ANNOTATIONS
  • ./bucket-name-images/platform/date/ID/imageType/images/ANNOTATIONS/unlabled/annotations-Dir0001.csv
  • ./bucket-name-images/platform/date/ID/imageType/images/ANNOTATIONS/unlabled/annotations-Dir0002.csv
  • ./bucket-name-images/platform/date/ID/imageType/images/ANNOTATIONS/unlabled/annotations-Dir0003.csv

TODO:

• Modify the current setup to support this new structure. I would probably make it so you input the /images/ directory on girder and reference the ANNOTATIONS folder locally and it will go through and match the annotations and Directory name to the image sequence names.
• Add capability to clone videos to public folder for easier organization, or possibly clone to a specified folder.
• Add support for specific keys in the annotation files that will cause it to clone the dataset instead of uploading directly.
• Create an explanation video about how it works

For reference I also ran our poetry tox -e format ../samples/scripts and that's why the uploadScript.py has been modified

SmallUpload.mp4

[marySalvi]

I don't think the girder id is actually optional.

[BryonLewis]

That was leftover from the earlier version when I had the girder_id specified in global variables. I'll change it.

[marySalvi]

LGTM