Skip to content

Commit 1d33234

Browse files
KN4CK3Rwxiaoguangtboerger
authored
Add Package Registry (#16510)
* Added package store settings. * Added models. * Added generic package registry. * Added tests. * Added NuGet package registry. * Moved service index to api file. * Added NPM package registry. * Added Maven package registry. * Added PyPI package registry. * Summary is deprecated. * Changed npm name. * Sanitize project url. * Allow only scoped packages. * Added user interface. * Changed method name. * Added missing migration file. * Set page info. * Added documentation. * Added documentation links. * Fixed wrong error message. * Lint template files. * Fixed merge errors. * Fixed unit test storage path. * Switch to json module. * Added suggestions. * Added package webhook. * Add package api. * Fixed swagger file. * Fixed enum and comments. * Fixed NuGet pagination. * Print test names. * Added api tests. * Fixed access level. * Fix User unmarshal. * Added RubyGems package registry. * Fix lint. * Implemented io.Writer. * Added support for sha256/sha512 checksum files. * Improved maven-metadata.xml support. * Added support for symbol package uploads. * Added tests. * Added overview docs. * Added npm dependencies and keywords. * Added no-packages information. * Display file size. * Display asset count. * Fixed filter alignment. * Added package icons. * Formatted instructions. * Allow anonymous package downloads. * Fixed comments. * Fixed postgres test. * Moved file. * Moved models to models/packages. * Use correct error response format per client. * Use simpler search form. * Fixed IsProd. * Restructured data model. * Prevent empty filename. * Fix swagger. * Implemented user/org registry. * Implemented UI. * Use GetUserByIDCtx. * Use table for dependencies. * make svg * Added support for unscoped npm packages. * Add support for npm dist tags. * Added tests for npm tags. * Unlink packages if repository gets deleted. * Prevent user/org delete if a packages exist. * Use package unlink in repository service. * Added support for composer packages. * Restructured package docs. * Added missing tests. * Fixed generic content page. * Fixed docs. * Fixed swagger. * Added missing type. * Fixed ambiguous column. * Organize content store by sha256 hash. * Added admin package management. * Added support for sorting. * Add support for multiple identical versions/files. * Added missing repository unlink. * Added file properties. * make fmt * lint * Added Conan package registry. * Updated docs. * Unify package names. * Added swagger enum. * Use longer TEXT column type. * Removed version composite key. * Merged package and container registry. * Removed index. * Use dedicated package router. * Moved files to new location. * Updated docs. * Fixed JOIN order. * Fixed GROUP BY statement. * Fixed GROUP BY #2. * Added symbol server support. * Added more tests. * Set NOT NULL. * Added setting to disable package registries. * Moved auth into service. * refactor * Use ctx everywhere. * Added package cleanup task. * Changed packages path. * Added container registry. * Refactoring * Updated comparison. * Fix swagger. * Fixed table order. * Use token auth for npm routes. * Enabled ReverseProxy auth. * Added packages link for orgs. * Fixed anonymous org access. * Enable copy button for setup instructions. * Merge error * Added suggestions. * Fixed merge. * Handle "generic". * Added link for TODO. * Added suggestions. * Changed temporary buffer filename. * Added suggestions. * Apply suggestions from code review Co-authored-by: Thomas Boerger <[email protected]> * Update docs/content/doc/packages/nuget.en-us.md Co-authored-by: wxiaoguang <[email protected]> Co-authored-by: Thomas Boerger <[email protected]>
1 parent 2bce1ea commit 1d33234

File tree

197 files changed

+18563
-55
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

197 files changed

+18563
-55
lines changed

‎custom/conf/app.example.ini

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1900,6 +1900,24 @@ PATH =
19001900
;; If CLEANUP_TYPE is set to PerWebhook, this is number of hook_task records to keep for a webhook (i.e. keep the most recent x deliveries).
19011901
;NUMBER_TO_KEEP = 10
19021902

1903+
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
1904+
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
1905+
;; Cleanup expired packages
1906+
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
1907+
;[cron.cleanup_packages]
1908+
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
1909+
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
1910+
;; Whether to enable the job
1911+
;ENABLED = true
1912+
;; Whether to always run at least once at start up time (if ENABLED)
1913+
;RUN_AT_START = true
1914+
;; Whether to emit notice on successful execution too
1915+
;NOTICE_ON_SUCCESS = false
1916+
;; Time interval for job to run
1917+
;SCHEDULE = @midnight
1918+
;; Unreferenced blobs created more than OLDER_THAN ago are subject to deletion
1919+
;OLDER_THAN = 24h
1920+
19031921
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
19041922
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
19051923
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
@@ -2221,6 +2239,18 @@ PATH =
22212239
;; Enable/Disable federation capabilities
22222240
; ENABLED = true
22232241

2242+
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
2243+
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
2244+
;[packages]
2245+
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
2246+
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
2247+
;;
2248+
;; Enable/Disable package registry capabilities
2249+
;ENABLED = true
2250+
;;
2251+
;; Path for chunked uploads. Defaults to APP_DATA_PATH + `tmp/package-upload`
2252+
;CHUNKED_UPLOAD_PATH = tmp/package-upload
2253+
22242254
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
22252255
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
22262256
;; default storage for attachments, lfs and avatars
@@ -2251,6 +2281,16 @@ PATH =
22512281
;; Where your lfs files reside, default is data/lfs.
22522282
;PATH = data/lfs
22532283

2284+
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
2285+
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
2286+
;; settings for packages, will override storage setting
2287+
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
2288+
;[storage.packages]
2289+
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
2290+
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
2291+
;; storage type
2292+
;STORAGE_TYPE = local
2293+
22542294
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
22552295
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
22562296
;; customize storage

‎docs/content/doc/advanced/config-cheat-sheet.en-us.md

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -856,7 +856,7 @@ Default templates for project boards:
856856
- `RUN_AT_START`: **true**: Run repository statistics check at start time.
857857
- `SCHEDULE`: **@midnight**: Cron syntax for scheduling repository statistics check.
858858

859-
### Cron - Cleanup hook_task Table (`cron.cleanup_hook_task_table`)
859+
#### Cron - Cleanup hook_task Table (`cron.cleanup_hook_task_table`)
860860

861861
- `ENABLED`: **true**: Enable cleanup hook_task job.
862862
- `RUN_AT_START`: **false**: Run cleanup hook_task at start time (if ENABLED).
@@ -865,6 +865,14 @@ Default templates for project boards:
865865
- `OLDER_THAN`: **168h**: If CLEANUP_TYPE is set to OlderThan, then any delivered hook_task records older than this expression will be deleted.
866866
- `NUMBER_TO_KEEP`: **10**: If CLEANUP_TYPE is set to PerWebhook, this is number of hook_task records to keep for a webhook (i.e. keep the most recent x deliveries).
867867

868+
#### Cron - Cleanup expired packages (`cron.cleanup_packages`)
869+
870+
- `ENABLED`: **true**: Enable cleanup expired packages job.
871+
- `RUN_AT_START`: **true**: Run job at start time (if ENABLED).
872+
- `NOTICE_ON_SUCCESS`: **false**: Notify every time this job runs.
873+
- `SCHEDULE`: **@midnight**: Cron syntax for the job.
874+
- `OLDER_THAN`: **24h**: Unreferenced package data created more than OLDER_THAN ago is subject to deletion.
875+
868876
#### Cron - Update Migration Poster ID (`cron.update_migration_poster_id`)
869877

870878
- `SCHEDULE`: **@midnight** : Interval as a duration between each synchronization, it will always attempt synchronization when the instance starts.
@@ -1077,6 +1085,11 @@ Task queue configuration has been moved to `queue.task`. However, the below conf
10771085

10781086
- `ENABLED`: **true**: Enable/Disable federation capabilities
10791087

1088+
## Packages (`packages`)
1089+
1090+
- `ENABLED`: **true**: Enable/Disable package registry capabilities
1091+
- `CHUNKED_UPLOAD_PATH`: **tmp/package-upload**: Path for chunked uploads. Defaults to `APP_DATA_PATH` + `tmp/package-upload`
1092+
10801093
## Mirror (`mirror`)
10811094

10821095
- `ENABLED`: **true**: Enables the mirror functionality. Set to **false** to disable all mirrors.

‎docs/content/doc/developers.en-us.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,6 @@ draft: false
88
menu:
99
sidebar:
1010
name: "Developers"
11-
weight: 50
11+
weight: 55
1212
identifier: "developers"
1313
---

‎docs/content/doc/developers.zh-tw.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,6 @@ draft: false
88
menu:
99
sidebar:
1010
name: "開發人員"
11-
weight: 50
11+
weight: 55
1212
identifier: "developers"
1313
---

‎docs/content/doc/features/comparison.en-us.md

Lines changed: 19 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -34,25 +34,25 @@ _Symbols used in table:_
3434
## General Features
3535

3636
| Feature | Gitea | Gogs | GitHub EE | GitLab CE | GitLab EE | BitBucket | RhodeCode CE |
37-
| ----------------------------------- | -------------------------------------------------- | ---- | --------- | --------- | --------- | -------------- | ------------ |
38-
| Open source and free | ✓ | ✓ | ✘ | ✓ | ✘ | ✘ | ✓ |
39-
| Low resource usage (RAM/CPU) | ✓ | ✓ | ✘ | ✘ | ✘ | ✘ | ✘ |
40-
| Multiple database support | ✓ | ✓ | ✘ | ⁄ | ⁄ | ✓ | ✓ |
41-
| Multiple OS support | ✓ | ✓ | ✘ | ✘ | ✘ | ✘ | ✓ |
42-
| Easy upgrade process | ✓ | ✓ | ✘ | ✓ | ✓ | ✘ | ✓ |
43-
| Markdown support | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
44-
| Orgmode support | ✓ | ✘ | ✓ | ✘ | ✘ | ✘ | ? |
45-
| CSV support | ✓ | ✘ | ✓ | ✘ | ✘ | ✓ | ? |
46-
| Third-party render tool support | ✓ | ✘ | ✘ | ✘ | ✘ | ✓ | ? |
47-
| Static Git-powered pages | [✘](https://github.com/go-gitea/gitea/issues/302) | ✘ | ✓ | ✓ | ✓ | ✘ | ✘ |
48-
| Integrated Git-powered wiki | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ (cloud only) | ✘ |
49-
| Deploy Tokens | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
50-
| Repository Tokens with write rights | ✓ | ✘ | ✓ | ✓ | ✓ | ✓ | ✓ |
51-
| Built-in Container Registry | [✘](https://github.com/go-gitea/gitea/issues/2316) | ✘ | ✘ | ✓ | ✓ | ✘ | ✘ |
52-
| External git mirroring | ✓ | ✓ | ✘ | ✘ | ✓ | ✓ | ✓ |
53-
| WebAuthn (2FA) | ✓ | ✘ | ✓ | ✓ | ✓ | ✓ | ? |
54-
| Built-in CI/CD | ✘ | ✘ | ✓ | ✓ | ✓ | ✘ | ✘ |
55-
| Subgroups: groups within groups | ✘ | ✘ | ✘ | ✓ | ✓ | ✘ | ✓ |
37+
| ----------------------------------- | ---------------------------------------------------| ---- | --------- | --------- | --------- | -------------- | ------------ |
38+
| Open source and free | ✓ | ✓ | ✘ | ✓ | ✘ | ✘ | ✓ |
39+
| Low resource usage (RAM/CPU) | ✓ | ✓ | ✘ | ✘ | ✘ | ✘ | ✘ |
40+
| Multiple database support | ✓ | ✓ | ✘ | ⁄ | ⁄ | ✓ | ✓ |
41+
| Multiple OS support | ✓ | ✓ | ✘ | ✘ | ✘ | ✘ | ✓ |
42+
| Easy upgrade process | ✓ | ✓ | ✘ | ✓ | ✓ | ✘ | ✓ |
43+
| Markdown support | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
44+
| Orgmode support | ✓ | ✘ | ✓ | ✘ | ✘ | ✘ | ? |
45+
| CSV support | ✓ | ✘ | ✓ | ✘ | ✘ | ✓ | ? |
46+
| Third-party render tool support | ✓ | ✘ | ✘ | ✘ | ✘ | ✓ | ? |
47+
| Static Git-powered pages | [✘](https://github.com/go-gitea/gitea/issues/302) | ✘ | ✓ | ✓ | ✓ | ✘ | ✘ |
48+
| Integrated Git-powered wiki | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ (cloud only) | ✘ |
49+
| Deploy Tokens | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
50+
| Repository Tokens with write rights | ✓ | ✘ | ✓ | ✓ | ✓ | ✓ | ✓ |
51+
| Built-in Package/Container Registry | ✓ | ✘ | ✓ | ✓ | ✓ | ✘ | ✘ |
52+
| External git mirroring | ✓ | ✓ | ✘ | ✘ | ✓ | ✓ | ✓ |
53+
| WebAuthn (2FA) | ✓ | ✘ | ✓ | ✓ | ✓ | ✓ | ? |
54+
| Built-in CI/CD | ✘ | ✘ | ✓ | ✓ | ✓ | ✘ | ✘ |
55+
| Subgroups: groups within groups | ✘ | ✘ | ✘ | ✓ | ✓ | ✘ | ✓ |
5656

5757
## Code management
5858

‎docs/content/doc/packages.en-us.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
---
2+
date: "2021-07-20T00:00:00+00:00"
3+
title: "Package Registry"
4+
slug: "packages"
5+
toc: false
6+
draft: false
7+
menu:
8+
sidebar:
9+
name: "Package Registry"
10+
weight: 45
11+
identifier: "packages"
12+
---

‎docs/content/doc/packages/composer.en-us.md

Lines changed: 120 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,120 @@
1+
---
2+
date: "2021-07-20T00:00:00+00:00"
3+
title: "Composer Packages Repository"
4+
slug: "packages/composer"
5+
draft: false
6+
toc: false
7+
menu:
8+
sidebar:
9+
parent: "packages"
10+
name: "Composer"
11+
weight: 10
12+
identifier: "composer"
13+
---
14+
15+
# Composer Packages Repository
16+
17+
Publish [Composer](https://getcomposer.org/) packages for your user or organization.
18+
19+
**Table of Contents**
20+
21+
{{< toc >}}
22+
23+
## Requirements
24+
25+
To work with the Composer package registry, you can use [Composer](https://getcomposer.org/download/) to consume and a HTTP upload client like `curl` to publish packages.
26+
27+
## Publish a package
28+
29+
To publish a Composer package perform a HTTP PUT operation with the package content in the request body.
30+
The package content must be the zipped PHP project with the `composer.json` file.
31+
You cannot publish a package if a package of the same name and version already exists. You must delete the existing package first.
32+
33+
```
34+
PUT https://gitea.example.com/api/packages/{owner}/composer
35+
```
36+
37+
| Parameter | Description |
38+
| ---------- | ----------- |
39+
| `owner` | The owner of the package. |
40+
41+
If the `composer.json` file does not contain a `version` property, you must provide it as a query parameter:
42+
43+
```
44+
PUT https://gitea.example.com/api/packages/{owner}/composer?version={x.y.z}
45+
```
46+
47+
Example request using HTTP Basic authentication:
48+
49+
```shell
50+
curl --user your_username:your_password_or_token \
51+
--upload-file path/to/project.zip \
52+
https://gitea.example.com/api/packages/testuser/composer
53+
```
54+
55+
Or specify the package version as query parameter:
56+
57+
```shell
58+
curl --user your_username:your_password_or_token \
59+
--upload-file path/to/project.zip \
60+
https://gitea.example.com/api/packages/testuser/composer?version=1.0.3
61+
```
62+
63+
The server responds with the following HTTP Status codes.
64+
65+
| HTTP Status Code | Meaning |
66+
| ----------------- | ------- |
67+
| `201 Created` | The package has been published. |
68+
| `400 Bad Request` | The package name and/or version are invalid or a package with the same name and version already exist. |
69+
70+
## Configuring the package registry
71+
72+
To register the package registry you need to add it to the Composer `config.json` file (which can usually be found under `<user-home-dir>/.composer/config.json`):
73+
74+
```json
75+
{
76+
"repositories": [{
77+
"type": "composer",
78+
"url": "https://gitea.example.com/api/packages/{owner}/composer"
79+
}
80+
]
81+
}
82+
```
83+
84+
To access the package registry using credentials, you must specify them in the `auth.json` file as follows:
85+
86+
```json
87+
{
88+
"http-basic": {
89+
"gitea.example.com": {
90+
"username": "{username}",
91+
"password": "{password}"
92+
}
93+
}
94+
}
95+
```
96+
97+
| Parameter | Description |
98+
| ---------- | ----------- |
99+
| `owner` | The owner of the package. |
100+
| `username` | Your Gitea username. |
101+
| `password` | Your Gitea password or a personal access token. |
102+
103+
## Install a package
104+
105+
To install a package from the package registry, execute the following command:
106+
107+
```shell
108+
composer require {package_name}
109+
```
110+
111+
Optional you can specify the package version:
112+
113+
```shell
114+
composer require {package_name}:{package_version}
115+
```
116+
117+
| Parameter | Description |
118+
| ----------------- | ----------- |
119+
| `package_name` | The package name. |
120+
| `package_version` | The package version. |

‎docs/content/doc/packages/conan.en-us.md

Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
---
2+
date: "2021-07-20T00:00:00+00:00"
3+
title: "Conan Packages Repository"
4+
slug: "packages/conan"
5+
draft: false
6+
toc: false
7+
menu:
8+
sidebar:
9+
parent: "packages"
10+
name: "Conan"
11+
weight: 20
12+
identifier: "conan"
13+
---
14+
15+
# Conan Packages Repository
16+
17+
Publish [Conan](https://conan.io/) packages for your user or organization.
18+
19+
**Table of Contents**
20+
21+
{{< toc >}}
22+
23+
## Requirements
24+
25+
To work with the Conan package registry, you need to use the [conan](https://conan.io/downloads.html) command line tool to consume and publish packages.
26+
27+
## Configuring the package registry
28+
29+
To register the package registry you need to configure a new Conan remote:
30+
31+
```shell
32+
conan remote add {remote} https://gitea.example.com/api/packages/{owner}/conan
33+
conan user --remote {remote} --password {password} {username}
34+
```
35+
36+
| Parameter | Description |
37+
| -----------| ----------- |
38+
| `remote` | The remote name. |
39+
| `username` | Your Gitea username. |
40+
| `password` | Your Gitea password or a personal access token. |
41+
| `owner` | The owner of the package. |
42+
43+
For example:
44+
45+
```shell
46+
conan remote add gitea https://gitea.example.com/api/packages/testuser/conan
47+
conan user --remote gitea --password password123 testuser
48+
```
49+
50+
## Publish a package
51+
52+
Publish a Conan package by running the following command:
53+
54+
```shell
55+
conan upload --remote={remote} {recipe}
56+
```
57+
58+
| Parameter | Description |
59+
| ----------| ----------- |
60+
| `remote` | The remote name. |
61+
| `recipe` | The recipe to upload. |
62+
63+
For example:
64+
65+
```shell
66+
conan upload --remote=gitea ConanPackage/1.2@gitea/final
67+
```
68+
69+
The Gitea Conan package registry has full [revision](https://docs.conan.io/en/latest/versioning/revisions.html) support.
70+
71+
## Install a package
72+
73+
To install a Conan package from the package registry, execute the following command:
74+
75+
```shell
76+
conan install --remote={remote} {recipe}
77+
```
78+
79+
| Parameter | Description |
80+
| ----------| ----------- |
81+
| `remote` | The remote name. |
82+
| `recipe` | The recipe to download. |
83+
84+
For example:
85+
86+
```shell
87+
conan install --remote=gitea ConanPackage/1.2@gitea/final
88+
```
89+
90+
## Supported commands
91+
92+
```
93+
conan install
94+
conan get
95+
conan info
96+
conan search
97+
conan upload
98+
conan user
99+
conan download
100+
conan remove
101+
```

0 commit comments

Comments
 (0)