Skip to content

Commit

Permalink
Add Package Registry (go-gitea#16510)
Browse files Browse the repository at this point in the history
* Added package store settings.

* Added models.

* Added generic package registry.

* Added tests.

* Added NuGet package registry.

* Moved service index to api file.

* Added NPM package registry.

* Added Maven package registry.

* Added PyPI package registry.

* Summary is deprecated.

* Changed npm name.

* Sanitize project url.

* Allow only scoped packages.

* Added user interface.

* Changed method name.

* Added missing migration file.

* Set page info.

* Added documentation.

* Added documentation links.

* Fixed wrong error message.

* Lint template files.

* Fixed merge errors.

* Fixed unit test storage path.

* Switch to json module.

* Added suggestions.

* Added package webhook.

* Add package api.

* Fixed swagger file.

* Fixed enum and comments.

* Fixed NuGet pagination.

* Print test names.

* Added api tests.

* Fixed access level.

* Fix User unmarshal.

* Added RubyGems package registry.

* Fix lint.

* Implemented io.Writer.

* Added support for sha256/sha512 checksum files.

* Improved maven-metadata.xml support.

* Added support for symbol package uploads.

* Added tests.

* Added overview docs.

* Added npm dependencies and keywords.

* Added no-packages information.

* Display file size.

* Display asset count.

* Fixed filter alignment.

* Added package icons.

* Formatted instructions.

* Allow anonymous package downloads.

* Fixed comments.

* Fixed postgres test.

* Moved file.

* Moved models to models/packages.

* Use correct error response format per client.

* Use simpler search form.

* Fixed IsProd.

* Restructured data model.

* Prevent empty filename.

* Fix swagger.

* Implemented user/org registry.

* Implemented UI.

* Use GetUserByIDCtx.

* Use table for dependencies.

* make svg

* Added support for unscoped npm packages.

* Add support for npm dist tags.

* Added tests for npm tags.

* Unlink packages if repository gets deleted.

* Prevent user/org delete if a packages exist.

* Use package unlink in repository service.

* Added support for composer packages.

* Restructured package docs.

* Added missing tests.

* Fixed generic content page.

* Fixed docs.

* Fixed swagger.

* Added missing type.

* Fixed ambiguous column.

* Organize content store by sha256 hash.

* Added admin package management.

* Added support for sorting.

* Add support for multiple identical versions/files.

* Added missing repository unlink.

* Added file properties.

* make fmt

* lint

* Added Conan package registry.

* Updated docs.

* Unify package names.

* Added swagger enum.

* Use longer TEXT column type.

* Removed version composite key.

* Merged package and container registry.

* Removed index.

* Use dedicated package router.

* Moved files to new location.

* Updated docs.

* Fixed JOIN order.

* Fixed GROUP BY statement.

* Fixed GROUP BY #2.

* Added symbol server support.

* Added more tests.

* Set NOT NULL.

* Added setting to disable package registries.

* Moved auth into service.

* refactor

* Use ctx everywhere.

* Added package cleanup task.

* Changed packages path.

* Added container registry.

* Refactoring

* Updated comparison.

* Fix swagger.

* Fixed table order.

* Use token auth for npm routes.

* Enabled ReverseProxy auth.

* Added packages link for orgs.

* Fixed anonymous org access.

* Enable copy button for setup instructions.

* Merge error

* Added suggestions.

* Fixed merge.

* Handle "generic".

* Added link for TODO.

* Added suggestions.

* Changed temporary buffer filename.

* Added suggestions.

* Apply suggestions from code review

Co-authored-by: Thomas Boerger <thomas@webhippie.de>

* Update docs/content/doc/packages/nuget.en-us.md

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Thomas Boerger <thomas@webhippie.de>
  • Loading branch information
3 people committed Mar 30, 2022
1 parent 2bce1ea commit 1d33234
Show file tree
Hide file tree
Showing 197 changed files with 18,563 additions and 55 deletions.
40 changes: 40 additions & 0 deletions custom/conf/app.example.ini
Original file line number Diff line number Diff line change
Expand Up @@ -1900,6 +1900,24 @@ PATH =
;; If CLEANUP_TYPE is set to PerWebhook, this is number of hook_task records to keep for a webhook (i.e. keep the most recent x deliveries).
;NUMBER_TO_KEEP = 10

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Cleanup expired packages
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;[cron.cleanup_packages]
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Whether to enable the job
;ENABLED = true
;; Whether to always run at least once at start up time (if ENABLED)
;RUN_AT_START = true
;; Whether to emit notice on successful execution too
;NOTICE_ON_SUCCESS = false
;; Time interval for job to run
;SCHEDULE = @midnight
;; Unreferenced blobs created more than OLDER_THAN ago are subject to deletion
;OLDER_THAN = 24h

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
Expand Down Expand Up @@ -2221,6 +2239,18 @@ PATH =
;; Enable/Disable federation capabilities
; ENABLED = true

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;[packages]
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;
;; Enable/Disable package registry capabilities
;ENABLED = true
;;
;; Path for chunked uploads. Defaults to APP_DATA_PATH + `tmp/package-upload`
;CHUNKED_UPLOAD_PATH = tmp/package-upload

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; default storage for attachments, lfs and avatars
Expand Down Expand Up @@ -2251,6 +2281,16 @@ PATH =
;; Where your lfs files reside, default is data/lfs.
;PATH = data/lfs

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; settings for packages, will override storage setting
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;[storage.packages]
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; storage type
;STORAGE_TYPE = local

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; customize storage
Expand Down
15 changes: 14 additions & 1 deletion docs/content/doc/advanced/config-cheat-sheet.en-us.md
Original file line number Diff line number Diff line change
Expand Up @@ -856,7 +856,7 @@ Default templates for project boards:
- `RUN_AT_START`: **true**: Run repository statistics check at start time.
- `SCHEDULE`: **@midnight**: Cron syntax for scheduling repository statistics check.

### Cron - Cleanup hook_task Table (`cron.cleanup_hook_task_table`)
#### Cron - Cleanup hook_task Table (`cron.cleanup_hook_task_table`)

- `ENABLED`: **true**: Enable cleanup hook_task job.
- `RUN_AT_START`: **false**: Run cleanup hook_task at start time (if ENABLED).
Expand All @@ -865,6 +865,14 @@ Default templates for project boards:
- `OLDER_THAN`: **168h**: If CLEANUP_TYPE is set to OlderThan, then any delivered hook_task records older than this expression will be deleted.
- `NUMBER_TO_KEEP`: **10**: If CLEANUP_TYPE is set to PerWebhook, this is number of hook_task records to keep for a webhook (i.e. keep the most recent x deliveries).

#### Cron - Cleanup expired packages (`cron.cleanup_packages`)

- `ENABLED`: **true**: Enable cleanup expired packages job.
- `RUN_AT_START`: **true**: Run job at start time (if ENABLED).
- `NOTICE_ON_SUCCESS`: **false**: Notify every time this job runs.
- `SCHEDULE`: **@midnight**: Cron syntax for the job.
- `OLDER_THAN`: **24h**: Unreferenced package data created more than OLDER_THAN ago is subject to deletion.

#### Cron - Update Migration Poster ID (`cron.update_migration_poster_id`)

- `SCHEDULE`: **@midnight** : Interval as a duration between each synchronization, it will always attempt synchronization when the instance starts.
Expand Down Expand Up @@ -1077,6 +1085,11 @@ Task queue configuration has been moved to `queue.task`. However, the below conf

- `ENABLED`: **true**: Enable/Disable federation capabilities

## Packages (`packages`)

- `ENABLED`: **true**: Enable/Disable package registry capabilities
- `CHUNKED_UPLOAD_PATH`: **tmp/package-upload**: Path for chunked uploads. Defaults to `APP_DATA_PATH` + `tmp/package-upload`

## Mirror (`mirror`)

- `ENABLED`: **true**: Enables the mirror functionality. Set to **false** to disable all mirrors.
Expand Down
2 changes: 1 addition & 1 deletion docs/content/doc/developers.en-us.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,6 @@ draft: false
menu:
sidebar:
name: "Developers"
weight: 50
weight: 55
identifier: "developers"
---
2 changes: 1 addition & 1 deletion docs/content/doc/developers.zh-tw.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,6 @@ draft: false
menu:
sidebar:
name: "開發人員"
weight: 50
weight: 55
identifier: "developers"
---
38 changes: 19 additions & 19 deletions docs/content/doc/features/comparison.en-us.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,25 +34,25 @@ _Symbols used in table:_
## General Features

| Feature | Gitea | Gogs | GitHub EE | GitLab CE | GitLab EE | BitBucket | RhodeCode CE |
| ----------------------------------- | -------------------------------------------------- | ---- | --------- | --------- | --------- | -------------- | ------------ |
| Open source and free | |||||||
| Low resource usage (RAM/CPU) | |||||||
| Multiple database support | |||||||
| Multiple OS support | |||||||
| Easy upgrade process | |||||||
| Markdown support | |||||||
| Orgmode support | |||||| ? |
| CSV support | |||||| ? |
| Third-party render tool support | |||||| ? |
| Static Git-powered pages | [](https://github.com/go-gitea/gitea/issues/302) |||||||
| Integrated Git-powered wiki | ||||| ✓ (cloud only) ||
| Deploy Tokens | |||||||
| Repository Tokens with write rights | |||||||
| Built-in Container Registry | [](https://github.com/go-gitea/gitea/issues/2316) || |||||
| External git mirroring | |||||||
| WebAuthn (2FA) | |||||| ? |
| Built-in CI/CD | |||||||
| Subgroups: groups within groups | |||||||
| ----------------------------------- | ---------------------------------------------------| ---- | --------- | --------- | --------- | -------------- | ------------ |
| Open source and free ||||||||
| Low resource usage (RAM/CPU) ||||||||
| Multiple database support ||||||||
| Multiple OS support ||||||||
| Easy upgrade process ||||||||
| Markdown support ||||||||
| Orgmode support ||||||| ? |
| CSV support ||||||| ? |
| Third-party render tool support ||||||| ? |
| Static Git-powered pages | [](https://github.com/go-gitea/gitea/issues/302) |||||||
| Integrated Git-powered wiki |||||| ✓ (cloud only) ||
| Deploy Tokens ||||||||
| Repository Tokens with write rights ||||||||
| Built-in Package/Container Registry | || |||||
| External git mirroring ||||||||
| WebAuthn (2FA) ||||||| ? |
| Built-in CI/CD ||||||||
| Subgroups: groups within groups ||||||||

## Code management

Expand Down
12 changes: 12 additions & 0 deletions docs/content/doc/packages.en-us.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
date: "2021-07-20T00:00:00+00:00"
title: "Package Registry"
slug: "packages"
toc: false
draft: false
menu:
sidebar:
name: "Package Registry"
weight: 45
identifier: "packages"
---
120 changes: 120 additions & 0 deletions docs/content/doc/packages/composer.en-us.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
---
date: "2021-07-20T00:00:00+00:00"
title: "Composer Packages Repository"
slug: "packages/composer"
draft: false
toc: false
menu:
sidebar:
parent: "packages"
name: "Composer"
weight: 10
identifier: "composer"
---

# Composer Packages Repository

Publish [Composer](https://getcomposer.org/) packages for your user or organization.

**Table of Contents**

{{< toc >}}

## Requirements

To work with the Composer package registry, you can use [Composer](https://getcomposer.org/download/) to consume and a HTTP upload client like `curl` to publish packages.

## Publish a package

To publish a Composer package perform a HTTP PUT operation with the package content in the request body.
The package content must be the zipped PHP project with the `composer.json` file.
You cannot publish a package if a package of the same name and version already exists. You must delete the existing package first.

```
PUT https://gitea.example.com/api/packages/{owner}/composer
```

| Parameter | Description |
| ---------- | ----------- |
| `owner` | The owner of the package. |

If the `composer.json` file does not contain a `version` property, you must provide it as a query parameter:

```
PUT https://gitea.example.com/api/packages/{owner}/composer?version={x.y.z}
```

Example request using HTTP Basic authentication:

```shell
curl --user your_username:your_password_or_token \
--upload-file path/to/project.zip \
https://gitea.example.com/api/packages/testuser/composer
```

Or specify the package version as query parameter:

```shell
curl --user your_username:your_password_or_token \
--upload-file path/to/project.zip \
https://gitea.example.com/api/packages/testuser/composer?version=1.0.3
```

The server responds with the following HTTP Status codes.

| HTTP Status Code | Meaning |
| ----------------- | ------- |
| `201 Created` | The package has been published. |
| `400 Bad Request` | The package name and/or version are invalid or a package with the same name and version already exist. |

## Configuring the package registry

To register the package registry you need to add it to the Composer `config.json` file (which can usually be found under `<user-home-dir>/.composer/config.json`):

```json
{
"repositories": [{
"type": "composer",
"url": "https://gitea.example.com/api/packages/{owner}/composer"
}
]
}
```

To access the package registry using credentials, you must specify them in the `auth.json` file as follows:

```json
{
"http-basic": {
"gitea.example.com": {
"username": "{username}",
"password": "{password}"
}
}
}
```

| Parameter | Description |
| ---------- | ----------- |
| `owner` | The owner of the package. |
| `username` | Your Gitea username. |
| `password` | Your Gitea password or a personal access token. |

## Install a package

To install a package from the package registry, execute the following command:

```shell
composer require {package_name}
```

Optional you can specify the package version:

```shell
composer require {package_name}:{package_version}
```

| Parameter | Description |
| ----------------- | ----------- |
| `package_name` | The package name. |
| `package_version` | The package version. |
101 changes: 101 additions & 0 deletions docs/content/doc/packages/conan.en-us.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
---
date: "2021-07-20T00:00:00+00:00"
title: "Conan Packages Repository"
slug: "packages/conan"
draft: false
toc: false
menu:
sidebar:
parent: "packages"
name: "Conan"
weight: 20
identifier: "conan"
---

# Conan Packages Repository

Publish [Conan](https://conan.io/) packages for your user or organization.

**Table of Contents**

{{< toc >}}

## Requirements

To work with the Conan package registry, you need to use the [conan](https://conan.io/downloads.html) command line tool to consume and publish packages.

## Configuring the package registry

To register the package registry you need to configure a new Conan remote:

```shell
conan remote add {remote} https://gitea.example.com/api/packages/{owner}/conan
conan user --remote {remote} --password {password} {username}
```

| Parameter | Description |
| -----------| ----------- |
| `remote` | The remote name. |
| `username` | Your Gitea username. |
| `password` | Your Gitea password or a personal access token. |
| `owner` | The owner of the package. |

For example:

```shell
conan remote add gitea https://gitea.example.com/api/packages/testuser/conan
conan user --remote gitea --password password123 testuser
```

## Publish a package

Publish a Conan package by running the following command:

```shell
conan upload --remote={remote} {recipe}
```

| Parameter | Description |
| ----------| ----------- |
| `remote` | The remote name. |
| `recipe` | The recipe to upload. |

For example:

```shell
conan upload --remote=gitea ConanPackage/1.2@gitea/final
```

The Gitea Conan package registry has full [revision](https://docs.conan.io/en/latest/versioning/revisions.html) support.

## Install a package

To install a Conan package from the package registry, execute the following command:

```shell
conan install --remote={remote} {recipe}
```

| Parameter | Description |
| ----------| ----------- |
| `remote` | The remote name. |
| `recipe` | The recipe to download. |

For example:

```shell
conan install --remote=gitea ConanPackage/1.2@gitea/final
```

## Supported commands

```
conan install
conan get
conan info
conan search
conan upload
conan user
conan download
conan remove
```
Loading

0 comments on commit 1d33234

Please sign in to comment.