Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release: 1.3.0 #86

Open
wants to merge 93 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
93 commits
Select commit Hold shift + click to select a range
f2fdeed
Add support for include on get_asset_children()
jhodges10 Jun 8, 2021
49cbb28
Convert build_asset_info to private method
jhodges10 Jun 8, 2021
c3b7dea
Add missing kwargs for pagination
jhodges10 Jun 8, 2021
ce3cf07
(chore): Add pull request template
jhodges10 Jul 7, 2021
f6ba232
Introduce helper function class (#76)
jhodges10 Jul 31, 2021
ffaa6be
Add support for library search (#78)
jhodges10 Jul 31, 2021
de74b81
Inmproved download functionality (#79)
jhodges10 Jul 31, 2021
07a28d0
Introduce telemetry, status bars, and re-factor
jhodges10 Aug 3, 2021
4908192
Update Readme
jhodges10 Aug 3, 2021
29dc0ce
Fix all the merge conflict issues
jhodges10 Aug 3, 2021
32efa1d
Add functions for fetching tree and downloading entire projects
jhodges10 Aug 3, 2021
4c23e4c
Fix more issues post-merge
jhodges10 Aug 3, 2021
dea3841
Fix some more issues after merging
jhodges10 Aug 3, 2021
d1daf39
Add CLI for uploading/downloading assets
jhodges10 Aug 3, 2021
f707168
Fix integration test
jhodges10 Aug 3, 2021
f1defd8
Fix missing creation of downloads directory
jhodges10 Aug 3, 2021
9a20758
Add docs
jhodges10 Aug 3, 2021
8f9dc6f
Resolve conflicts post-merge
jhodges10 Aug 3, 2021
10261c0
Fix CI config
jhodges10 Aug 3, 2021
97fefec
Restore streaming download (RAM friendly)
jhodges10 Aug 3, 2021
66e72ec
Remove merge straggler
jhodges10 Aug 3, 2021
3649ce6
Refactor examples, and add one more for RBC
jhodges10 Aug 3, 2021
1c8a201
Update range-based comment example
jhodges10 Aug 3, 2021
586c29c
Refactor for transfer class
jhodges10 Aug 3, 2021
a81a9b5
Fix merge conflict
jhodges10 Aug 3, 2021
da6964d
Add more documentation
jhodges10 Aug 3, 2021
9342ff4
Clean-up unused imports
jhodges10 Aug 3, 2021
439c546
Formatting using black
jhodges10 Aug 3, 2021
8f21723
Fix some more issues and get recursive uploads working
jhodges10 Aug 7, 2021
ff1f86b
Solve for missing Segment key in ENV
jhodges10 Aug 7, 2021
e094d7f
Fix the retry mechanism
jhodges10 Aug 10, 2021
a202220
Add get_accounts() function
jhodges10 Aug 10, 2021
6e5225b
Move retryable statuses to constants.py
jhodges10 Aug 10, 2021
e172a57
Add upload progress bar for CLI
jhodges10 Aug 14, 2021
cb84310
Fix doc strings in projects.py
jhodges10 Sep 9, 2021
11865bb
Fix typo in version_stack detection
jhodges10 Nov 22, 2021
632b8d1
Introduce Config for simpler default const storage
jhodges10 Dec 21, 2021
1a3c632
Fix docstring formatting
jhodges10 Dec 21, 2021
9e67650
Run format
jhodges10 Dec 21, 2021
7207a19
Fix the API host in Config
jhodges10 Dec 21, 2021
00aa971
Don't print @Reference annotations
jhodges10 Dec 21, 2021
dfa18aa
Overhaul asset_scraper.py example
jhodges10 Dec 21, 2021
86ec593
Don't limit asset scraper
jhodges10 Dec 21, 2021
cdec461
Fix issue w/ _get_session()
jhodges10 Dec 21, 2021
005020f
More progress re-architecting transfer and FrameioDownloader
jhodges10 Dec 21, 2021
44b68d1
Remove progress bars and refactor AWSClient
jhodges10 Jan 4, 2022
910b4ed
Tweak docs build for CircleCI
jhodges10 Jan 4, 2022
b4ea321
Update language
jhodges10 Jan 4, 2022
19ced06
Add missing dependency
jhodges10 Jan 4, 2022
f654ad5
Fix tests for < py3.6.2
jhodges10 Jan 4, 2022
f4444b0
Fix f-string issue
jhodges10 Jan 4, 2022
e35dea3
Remove splats
jhodges10 Jan 4, 2022
2f2271c
Fix for py2
jhodges10 Jan 4, 2022
eab63de
Upstream dep update to more inclusive language
jhodges10 Jan 7, 2022
14b3fdd
Update recursive uploader code
jhodges10 Jan 11, 2022
08efeda
Fix collaborator and team manager functions
jhodges10 Feb 3, 2022
764823d
Add documentation builder action
jhodges10 Feb 4, 2022
4961366
Improve documentation, add type hints, start to break py2
jhodges10 Feb 4, 2022
09a35e8
Fix action trigger branch
jhodges10 Feb 4, 2022
03ae74d
Tweak github action config again
jhodges10 Feb 4, 2022
2cb8924
Directly assign requirements_path
jhodges10 Feb 4, 2022
751f743
Tweak requirements.txt path
jhodges10 Feb 4, 2022
224ad41
Fix requirements.txt installation directory
jhodges10 Feb 4, 2022
b8e4c26
Tweak config again
jhodges10 Feb 4, 2022
a88ee19
Try adding repository_path
jhodges10 Feb 4, 2022
43a2581
Disable telemetry for now
jhodges10 Feb 4, 2022
028885c
Trying again w/ simpler requirements_path
jhodges10 Feb 4, 2022
29a3d02
Trying new syntax
jhodges10 Feb 4, 2022
d6f3242
Trying the same thing again
jhodges10 Feb 4, 2022
0c239e9
Apply inputs to the correct action
jhodges10 Feb 4, 2022
fe539b6
Rename docs workflow -> documentation
jhodges10 Feb 4, 2022
c4615a0
Disable python 2.x builds
jhodges10 Feb 4, 2022
5c77bb6
Replace .format() w/ f-strings
jhodges10 Feb 4, 2022
03a1029
Force docs to develop branch
jhodges10 Feb 4, 2022
edd8c9c
Fix requests retry config
jhodges10 Feb 4, 2022
8e09f89
Remove < py3.6 compat
jhodges10 Feb 4, 2022
130b083
Bump version: 1.2.0 → 2.0.0
jhodges10 Feb 4, 2022
11d4102
Add types, improve docstrings, add asset versioning
jhodges10 Feb 4, 2022
85df47a
Fix assets.copy
jhodges10 Feb 4, 2022
8780d7e
Fix formatting of MB/s
jhodges10 Feb 4, 2022
a87b5a4
Add some more types
jhodges10 Feb 4, 2022
0ecb812
Add .nojekyll file to fix github pages problem
jhodges10 Feb 4, 2022
e32c869
Move .nojekyll
jhodges10 Feb 4, 2022
d6dbaa4
Try to fix the github pages issue
jhodges10 Feb 4, 2022
556b835
Fix library search example
jhodges10 Feb 4, 2022
f712baf
Run make format
jhodges10 Feb 4, 2022
6eba4ba
Don't run CI on gh-pages
jhodges10 Feb 7, 2022
2598820
Tweak python version matrix for testing
jhodges10 Feb 7, 2022
ec3d25d
Update comment_scraper.py (#94)
jhurtadosandoval Feb 28, 2023
cfdda3d
Cleanup unused functions
jhodges10 May 23, 2024
3c0b260
Bump circleci python orb
jhodges10 May 23, 2024
e352505
Tweak python version matrix for testing
jhodges10 May 23, 2024
01867dc
Little fixes
jhodges10 May 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion .bumpversion.cfg
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[bumpversion]
current_version = 1.1.0
current_version = 2.0.0
commit = True
tag = True

Expand Down
62 changes: 11 additions & 51 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
@@ -1,27 +1,34 @@
version: 2.1

orbs:
python: circleci/python@0.2.1
python: circleci/python@1.0.0
win: circleci/windows@2.2.0

workflows:
version: 2
build_test_deploy:
jobs:
- build
- build:
filters:
branches:
ignore:
- gh-pages

- test_integration:
requires:
- build
filters:
branches:
ignore:
- gh-pages
matrix:
parameters:
python-version: ["2.7.16", "2.7.18", "3.5.10", "3.6.5", "3.7.7", "3.8.6", "3.9.3", "latest"]
python-version: ["3.6.5", "3.7.7", "3.8.6", "3.9.3", "3.9.6", "3.9.9", "latest"]

- hold:
type: approval
requires:
- test_integration

filters:
branches:
only:
Expand All @@ -33,36 +40,18 @@ workflows:
requires:
- hold

# upload_test:
# triggers:
# - schedule:
# cron: "0,30 * * * *"
# filters:
# branches:
# only:
# - jh/use-xxhash-for-integration-test

# jobs:
# - build

# - upload_test_job:
# requires:
# - build

jobs:
build:
docker:
- image: circleci/python:latest
steps:
- checkout:
name: Checkout Git

- run:
name: Build Package
command: |
echo -e "Running sdist"
python setup.py sdist

- persist_to_workspace:
root: /home/circleci/project/
paths:
Expand All @@ -73,68 +62,39 @@ jobs:
parameters:
python-version:
type: string

docker:
- image: circleci/python:<< parameters.python-version >>

steps:
- attach_workspace:
at: /tmp/artifact
name: Attach build artifact

- run:
name: Install package
command: |
pip install --user '/tmp/artifact'

- run:
name: Run integration test
command: |
python /tmp/artifact/tests/integration.py

upload_test_job:
description: Upload test
docker:
- image: circleci/python:latest

steps:
- attach_workspace:
at: /tmp/artifact
name: Attach build artifact

- run:
name: Install package
command: |
pip install '/tmp/artifact'

- run:
name: Run integration test
command: |
python /tmp/artifact/tests/integration.py


deploy:
docker:
- image: circleci/python:latest

steps:
- attach_workspace:
at: /tmp/artifact
name: Attach build artifact

- run:
name: Install dependencies
command: |
pip install setuptools wheel twine

- run:
name: init .pypirc
command: |
cd /tmp/artifact
echo -e "[pypi]" >> ~/.pypirc
echo -e "username = $TWINE_USERNAME" >> ~/.pypirc
echo -e "password = $TWINE_PASSWORD" >> ~/.pypirc

- run:
name: Upload to pypi
command: |
Expand Down
13 changes: 13 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
## [DEVREL-XXXX]

### Description:
Please provide a short description of what this PR does

### Depends on:
- Does this PR depend on any other ones?

### Includes changes from:
- Does this PR includ changes from another PR?

### I'd like feedback on:
- What would you like feedback on?
24 changes: 24 additions & 0 deletions .github/workflows/documentation.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
name: Documentation
on:
push:
branches:
- develop
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/setup-python@v2
- uses: actions/checkout@master
with:
fetch-depth: 0 # otherwise, you will failed to push refs to dest repo
ref: develop
- name: Build and Commit
uses: sphinx-notes/pages@v2
with:
documentation_path: './docs'
requirements_path: './docs/requirements.txt'
- name: Push changes
uses: ad-m/github-push-action@master
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
branch: gh-pages
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -108,3 +108,6 @@ venv.bak/
Pipfile
Pipfile.lock
.vscode/launch.json
.vscode/settings.json

pyproject.toml
3 changes: 3 additions & 0 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"python.formatting.provider": "black"
}
34 changes: 34 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
FROM python:3.8.6-slim-buster as deps
# Set work directory
WORKDIR /home/speedtest

# Copy files
COPY Pipfile .
COPY Pipfile.lock .

# Install pipenv
RUN pip install pipenv

FROM deps as installer
# Set work directory
WORKDIR /home/speedtest

# Install deps
RUN pipenv install --system --deploy --ignore-pipfile

# Copy over the other pieces
COPY frameioclient frameioclient
COPY setup.py .
COPY README.md .

# Install the local frameioclient
RUN pipenv install -e . --skip-lock

# Copy over scripts and tests
COPY scripts scripts
COPY tests tests

ENV SEGMENT_WRITE_KEY=

FROM installer as runtime
ENTRYPOINT [ "pipenv", "run", "python", "scripts/benchmark/download.py" ]
21 changes: 21 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,24 @@ bump-patch:

clean:
find . -name "*.pyc" -exec rm -f {} \;

test:
cd tests && pipenv run python integration.py

package:
pipenv run python3 setup.py sdist bdist_wheel

build-docker:
docker build . -t benchmark

run-benchmark:
docker run -it -e $1 benchmark

format:
black frameioclient

view-docs:
cd docs && pip install -r requirements.txt && make dev

publish-docs:
cd docs && pip install -r requirements.txt && make jekyll && make publish
48 changes: 47 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,12 +24,58 @@ $ git clone https://github.com/frameio/python-frameio-client
$ pip install .
```

_Note: The Frame.io Python client may not work correctly in Python 3.8+_
### Developing
Install the package into your development environment and link to it by running the following:

```sh
pipenv install -e . -pre
```

## Documentation

[Frame.io API Documentation](https://developer.frame.io/docs)

### Use CLI
When you install this package, a cli tool called `fioctl` will also be installed to your environment.

**To upload a file or folder**
```sh
fioctl \
--token fio-u-YOUR_TOKEN_HERE \
--destination "YOUR TARGET FRAME.IO PROJECT OR FOLDER" \
--target "YOUR LOCAL SYSTEM DIRECTORY" \
--threads 8
```

**To download a file, project, or folder**
```sh
fioctl \
--token fio-u-YOUR_TOKEN_HERE \
--destination "YOUR LOCAL SYSTEM DIRECTORY" \
--target "YOUR TARGET FRAME.IO PROJECT OR FOLDER" \
--threads 2
```

### Links

**Sphinx Documentation**
- https://pythonhosted.org/sphinxcontrib-restbuilder/
- https://www.npmjs.com/package/rst-selector-parser
- https://sphinx-themes.org/sample-sites/furo/_sources/index.rst.txt
- https://developer.mantidproject.org/Standards/DocumentationGuideForDevs.html
- https://sublime-and-sphinx-guide.readthedocs.io/en/latest/code_blocks.html
- https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html
- https://stackoverflow.com/questions/64451966/python-sphinx-how-to-embed-code-into-a-docstring
- https://pythonhosted.org/an_example_pypi_project/sphinx.html

**Decorators**
- https://docs.python.org/3.7/library/functools.html
- https://realpython.com/primer-on-python-decorators/
- https://www.sphinx-doc.org/en/master/usage/quickstart.html
- https://www.geeksforgeeks.org/decorators-with-parameters-in-python/
- https://stackoverflow.com/questions/43544954/why-does-sphinx-autodoc-output-a-decorators-docstring-when-there-are-two-decora


## Usage

_Note: A valid token is required to make requests to Frame.io. Go to our [Developer Portal](https://developer.frame.io/) to get a token!_
Expand Down
36 changes: 36 additions & 0 deletions docs/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Minimal makefile for Sphinx documentation
#

# You can set these variables from the command line, and also
# from the environment for the first two.
SPHINXOPTS ?=
SPHINXBUILD ?= sphinx-build
SOURCEDIR = .
BUILDDIR = dist

# Put it first so that "make" without argument is like "make help".
help:
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

.PHONY: help Makefile

publish:
python publish.py

# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

jekyll:
sphinx-build -b jekyll . dist/markdown

rst:
sphinx-build -b rst . dist/rst

html:
sphinx-build -b html . dist/html

dev:
sphinx-autobuild -b html . _build/html

5 changes: 5 additions & 0 deletions docs/classes/assets.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
Assets
=========================

.. autoclass:: frameioclient.Asset
:members:
5 changes: 5 additions & 0 deletions docs/classes/comments.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
Comments
===================

.. autoclass:: frameioclient.Comment
:members:
12 changes: 12 additions & 0 deletions docs/classes/index.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
Classes
=====================

.. toctree::
users
assets
comments
logs
projects
teams
sharing
search
5 changes: 5 additions & 0 deletions docs/classes/logs.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
Audit Logs
===================

.. autoclass:: frameioclient.AuditLogs
:members:
5 changes: 5 additions & 0 deletions docs/classes/projects.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
Projects
===================

.. autoclass:: frameioclient.Project
:members: