Merge branch 'main' into private-comments

Author: Gusted

CHANGELOG.md

@@ -4,6 +4,105 @@ This changelog goes through all the changes that have been made in each release
 without substantial changes to our git log; to see the highlights of what has
 been added to each release, please refer to the [blog](https://blog.gitea.io).
 
+## [1.16.4](https://github.com/go-gitea/gitea/releases/tag/v1.16.4) - 2022-03-14
+
+* SECURITY
+  * Restrict email address validation (#17688) (#19085)
+  * Fix lfs bug (#19072) (#19080)
+* ENHANCEMENTS
+  * Improve SyncMirrors logging (#19045) (#19050)
+* BUGFIXES
+  * Refactor mirror code & fix `StartToMirror` (#18904) (#19075)
+  * Update the webauthn_credential_id_sequence in Postgres (#19048) (#19060)
+  * Prevent 500 when there is an error during new auth source post (#19041) (#19059)
+  * If rendering has failed due to a net.OpError stop rendering (attempt 2) (#19049) (#19056)
+  * Fix flag validation (#19046) (#19051)
+  * Add pam account authorization check (#19040) (#19047)
+  * Ignore missing comment for user notifications (#18954) (#19043)
+  * Set `rel="nofollow noindex"` on new issue links (#19023) (#19042)
+  * Upgrading binding package (#19034) (#19035)
+  * Don't show context cancelled errors in attribute reader (#19006) (#19027)
+  * Fix update hint bug (#18996) (#19002)
+* MISC
+  *  Fix potential assignee query for repo (#18994) (#18999)
+
+## [1.16.3](https://github.com/go-gitea/gitea/releases/tag/v1.16.3) - 2022-03-02
+
+* SECURITY
+ * Git backend ignore replace objects (#18979) (#18980) 
+* ENHANCEMENTS
+  * Adjust error for already locked db and prevent level db lock on malformed connstr (#18923) (#18938)
+* BUGFIXES
+  * Set max text height to prevent overflow (#18862) (#18977)
+  * Fix newAttachmentPaths deletion for DeleteRepository() (#18973) (#18974)
+  * Accounts with WebAuthn only (no TOTP) now exist ... fix code to handle that case (#18897) (#18964)
+  * Send 404 on `/{org}.gpg` (#18959) (#18962)
+  * Fix admin user list pagination (#18957) (#18960)
+  * Fix lfs management setting (#18947) (#18946)
+  * Fix login with email panic when email is not exist (#18942)
+  * Update go-org to v1.6.1 (#18932) (#18933)
+  * Fix `<strong>` html in translation (#18929) (#18931)
+  * Fix page and missing return on unadopted repos API (#18848) (#18927)
+  * Allow adminstrator teams members to see other teams (#18918) (#18919)
+  * Don't treat BOM escape sequence as hidden character. (#18909) (#18910)
+  * Correctly link URLs to users/repos with dashes, dots or underscores (… (#18908)
+  * Fix redirect when using lowercase repo name (#18775) (#18902)
+  * Fix migration v210 (#18893) (#18892)
+  * Fix team management UI (#18887) (18886)
+  * BeforeSourcePath should point to base commit (#18880) (#18799)
+* TRANSLATION
+  * Backport locales from master (#18944)
+* MISC
+  * Don't update email for organisation (#18905) (#18906)
+
+## [1.16.2](https://github.com/go-gitea/gitea/releases/tag/v1.16.2) - 2022-02-24
+
+* ENHANCEMENTS
+  * Show fullname on issue edits and gpg/ssh signing info (#18828)
+  * Immediately Hammer if second kill is sent (#18823) (#18826)
+  * Allow mermaid render error to wrap (#18791)
+* BUGFIXES
+  * Fix ldap user sync missed email in email_address table (#18786) (#18876) 
+  * Update assignees check to include any writing team and change org sidebar (#18680) (#18873)
+  * Don't report signal: killed errors in serviceRPC (#18850) (#18865)
+  * Fix bug where certain LDAP settings were reverted (#18859)
+  * Update go-org to 1.6.0 (#18824) (#18839)
+  * Fix login with email for ldap users (#18800) (#18836)
+  * Fix bug for get user by email (#18834)
+  * Fix panic in EscapeReader (#18820) (#18821)
+  * Fix ldap loginname (#18789) (#18804)
+  * Remove redundant call to UpdateRepoStats during migration (#18591) (#18794)
+  * In disk_channel queues synchronously push to disk on shutdown (#18415) (#18788)
+  * Fix template bug of LFS lock (#18784) (#18787)
+  * Attempt to fix the webauthn migration again - part 3 (#18770) (#18771)
+  * Send mail to issue/pr assignee/reviewer also when OnMention is set (#18707) (#18765)
+  * Fix a broken link in commits_list_small.tmpl (#18763) (#18764)
+  * Increase the size of the webauthn_credential credential_id field (#18739) (#18756)
+  * Prevent dangling GetAttribute calls (#18754) (#18755)
+  * Fix isempty detection of git repository (#18746) (#18750)
+  * Fix source code line highlighting on external tracker (#18729) (#18740)
+  * Prevent double encoding of branch names in delete branch (#18714) (#18738)
+  * Always set PullRequestWorkInProgressPrefixes in PrepareViewPullInfo (#18713) (#18737)
+  * Fix forked repositories missed tags (#18719) (#18735)
+  * Fix release typo (#18728) (#18731)
+  * Separate the details links of commit-statuses in headers (#18661) (#18730)
+  * Update object repo with the migrated repository (#18684) (#18726)
+  * Fix bug for version update hint (#18701) (#18705)
+  * Fix issue with docker-rootless shimming script (#18690) (#18699)
+  * Let `MinUnitAccessMode` return correct perm (#18675) (#18689)
+  * Prevent security failure due to bad APP_ID (#18678) (#18682)
+  * Restart zero worker if there is still work to do (#18658) (#18672)
+  * If rendering has failed due to a net.OpError stop rendering (#18642) (#18645)
+* TESTING
+  * Ensure git tag tests and others create test repos in tmpdir (#18447) (#18767)
+* BUILD
+  * Reduce CI go module downloads, add make targets (#18708, #18475, #18443) (#18741)
+* MISC
+  * Put buttons back in org dashboard (#18817) (#18825)
+  * Various Mermaid improvements (#18776) (#18780)
+  * C preprocessor colors improvement (#18671) (#18696)
+  * Fix the missing i18n key for update checker (#18646) (#18665)
+
 ## [1.16.1](https://github.com/go-gitea/gitea/releases/tag/v1.16.1) - 2022-02-06
 
 * SECURITY

cmd/cmd.go

@@ -31,7 +31,7 @@ func argsSet(c *cli.Context, args ...string) error {
 			return errors.New(a + " is not set")
 		}
 
-		if util.IsEmptyString(a) {
+		if util.IsEmptyString(c.String(a)) {
 			return errors.New(a + " is required")
 		}
 	}

contrib/init/openwrt/gitea

@@ -0,0 +1,35 @@
+#!/bin/sh /etc/rc.common
+
+USE_PROCD=1
+
+# PROCD_DEBUG=1
+
+START=90
+STOP=10
+
+PROG=/opt/gitea/gitea
+GITEA_WORK_DIR=/opt/gitea
+CONF_FILE=$GITEA_WORK_DIR/app.ini
+
+start_service(){
+    procd_open_instance gitea
+    procd_set_param env GITEA_WORK_DIR=$GITEA_WORK_DIR
+    procd_set_param env HOME=$GITEA_WORK_DIR
+    procd_set_param command $PROG web -c $CONF_FILE
+    procd_set_param file $CONF_FILE
+    procd_set_param user git
+    procd_set_param respawn ${respawn_threshold:-3600} ${respawn_timeout:-5} ${respawn_retry:-5} # respawn automatically if something died, be careful if you have an alternative process supervisor
+    procd_close_instance
+}
+
+start(){
+    service_start $PROG
+}
+
+stop(){
+    service_stop $PROG
+}
+
+reload(){
+    service_reload $PROG
+}

contrib/update_dependencies.sh

@@ -4,5 +4,5 @@ grep 'git' go.mod | grep '\.com' | grep -v indirect | grep -v replace | cut -f 2
   go get -u "$line"
   make vendor
   git add .
-  git commit -S -m "update $line"
+  git commit -m "update $line"
 done

contrib/upgrade.sh

@@ -1,6 +1,4 @@
 #!/usr/bin/env bash
-set -euo pipefail
-
 # This is an update script for gitea installed via the binary distribution
 # from dl.gitea.io on linux as systemd service. It performs a backup and updates
 # Gitea in place.
@@ -13,41 +11,83 @@ set -euo pipefail
 #   upgrade.sh 1.15.10
 #   giteahome=/opt/gitea giteaconf=$giteahome/app.ini upgrade.sh
 
+while true; do
+  case "$1" in
+    -v | --version ) ver="$2"; shift 2 ;;
+    -y | --yes ) no_confirm="yes"; shift ;;
+    --ignore-gpg) ignore_gpg="yes"; shift ;;
+    -- ) shift; break ;;
+    * ) break ;;
+  esac
+done
+
+set -euo pipefail
+
+
+function require {
+  for exe in "$@"; do
+    command -v "$exe" &>/dev/null || (echo "missing dependency '$exe'"; exit 1)
+  done
+}
+
+
+require curl xz sha256sum gpg
+
+if [[ -f /etc/os-release ]]; then
+  os_release=$(cat /etc/os-release)
+
+  if [[ "$os_release" =~ "OpenWrt" ]]; then
+    sudocmd="su"
+    service_start="/etc/init.d/gitea start"
+    service_stop="/etc/init.d/gitea stop"
+    service_status="/etc/init.d/gitea status"
+  else
+    require systemctl
+  fi
+fi
+
+
 # apply variables from environment
 : "${giteabin:="/usr/local/bin/gitea"}"
 : "${giteahome:="/var/lib/gitea"}"
 : "${giteaconf:="/etc/gitea/app.ini"}"
 : "${giteauser:="git"}"
 : "${sudocmd:="sudo"}"
 : "${arch:="linux-amd64"}"
+: "${service_start:="$sudocmd systemctl start gitea"}"
+: "${service_stop:="$sudocmd systemctl stop gitea"}"
+: "${service_status:="$sudocmd systemctl status gitea"}"
 : "${backupopts:=""}" # see `gitea dump --help` for available options
 
-function giteacmd {
-  "$sudocmd" --user "$giteauser" "$giteabin" --config "$giteaconf" --work-path "$giteahome" "$@"
-}
 
-function require {
-  for exe in "$@"; do
-    command -v "$exe" &>/dev/null || (echo "missing dependency '$exe'"; exit 1)
-  done
+function giteacmd {
+  if [[ $sudocmd = "su" ]]; then
+    "$sudocmd" - "$giteauser" -c "$giteabin" --config "$giteaconf" --work-path "$giteahome" "$@"
+  else
+    "$sudocmd" --user "$giteauser" "$giteabin" --config "$giteaconf" --work-path "$giteahome" "$@"
+  fi
 }
-require systemctl curl xz sha256sum gpg "$sudocmd"
 
 # select version to install
-if [[ -z "${1:-}" ]]; then
+if [[ -z "${ver:-}" ]]; then
   require jq
   giteaversion=$(curl --connect-timeout 10 -sL https://dl.gitea.io/gitea/version.json | jq -r .latest.version)
 else
-  giteaversion="$1"
+  giteaversion="$ver"
 fi
 
+
 # confirm update
-current=$(giteacmd --version | cut --delimiter=' ' --fields=3)
+current=$(giteacmd --version | cut -d ' ' -f 3)
 [[ "$current" == "$giteaversion" ]] && echo "$current is already installed, stopping." && exit 1
-echo "Make sure to read the changelog first: https://github.com/go-gitea/gitea/blob/main/CHANGELOG.md"
-echo "Are you ready to update Gitea from ${current} to ${giteaversion}? (y/N)"
-read -r confirm
-[[ "$confirm" == "y" ]] || [[ "$confirm" == "Y" ]] || exit 1
+if [[ -z "${no_confirm:-}"  ]]; then
+  echo "Make sure to read the changelog first: https://github.com/go-gitea/gitea/blob/main/CHANGELOG.md"
+  echo "Are you ready to update Gitea from ${current} to ${giteaversion}? (y/N)"
+  read -r confirm
+  [[ "$confirm" == "y" ]] || [[ "$confirm" == "Y" ]] || exit 1
+fi
+
+echo "Upgrading gitea from $current to $giteaversion ..."
 
 pushd "$(pwd)" &>/dev/null
 cd "$giteahome" # needed for gitea dump later
@@ -59,9 +99,11 @@ echo "Downloading $binurl..."
 curl --connect-timeout 10 --silent --show-error --fail --location -O "$binurl{,.sha256,.asc}"
 
 # validate checksum & gpg signature (exit script if error)
-sha256sum --check "${binname}.xz.sha256"
-gpg --keyserver keys.openpgp.org --recv 7C9E68152594688862D62AF62D9AE806EC1592E2
-gpg --verify "${binname}.xz.asc" "${binname}.xz" || { echo 'Signature does not match'; exit 1; }
+sha256sum -c "${binname}.xz.sha256"
+if [[ -z "${ignore_gpg:-}" ]]; then
+  gpg --keyserver keys.openpgp.org --recv 7C9E68152594688862D62AF62D9AE806EC1592E2
+  gpg --verify "${binname}.xz.asc" "${binname}.xz" || { echo 'Signature does not match'; exit 1; }
+fi
 rm "${binname}".xz.{sha256,asc}
 
 # unpack binary + make executable
@@ -72,12 +114,14 @@ chmod +x "$binname"
 # stop gitea, create backup, replace binary, restart gitea
 echo "Stopping gitea at $(date)"
 giteacmd manager flush-queues
-$sudocmd systemctl stop gitea
+$service_stop
 echo "Creating backup in $giteahome"
 giteacmd dump $backupopts
 echo "Updating binary at $giteabin"
-mv --force --backup "$binname" "$giteabin"
-$sudocmd systemctl start gitea
-$sudocmd systemctl status gitea
+cp -f "$giteabin" "$giteabin.bak" && mv -f "$binname" "$giteabin"
+$service_start
+$service_status
+
+echo "Upgrade to $giteaversion successful!"
 
 popd

custom/conf/app.example.ini

@@ -2125,6 +2125,8 @@ PATH =
 ;RENDER_COMMAND = "asciidoc --out-file=- -"
 ;; Don't pass the file on STDIN, pass the filename as argument instead.
 ;IS_INPUT_FILE = false
+; Don't filter html tags and attributes if true
+;DISABLE_SANITIZER = false
 
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

docs/config.yaml

@@ -18,7 +18,7 @@ params:
   description: Git with a cup of tea
   author: The Gitea Authors
   website: https://docs.gitea.io
-  version: 1.16.0
+  version: 1.16.4
   minGoVersion: 1.16
   goVersion: 1.17
   minNodeVersion: 12.17

docs/content/doc/advanced/config-cheat-sheet.en-us.md

@@ -1003,13 +1003,13 @@ IS_INPUT_FILE = false
    command. Multiple extensions needs a comma as splitter.
 - RENDER\_COMMAND: External command to render all matching extensions.
 - IS\_INPUT\_FILE: **false** Input is not a standard input but a file param followed `RENDER_COMMAND`.
+- DISABLE_SANITIZER: **false** Don't filter html tags and attributes if true. Don't change this to true except you know what that means.
 
 Two special environment variables are passed to the render command:
 - `GITEA_PREFIX_SRC`, which contains the current URL prefix in the `src` path tree. To be used as prefix for links.
 - `GITEA_PREFIX_RAW`, which contains the current URL prefix in the `raw` path tree. To be used as prefix for image paths.
 
-
-Gitea supports customizing the sanitization policy for rendered HTML. The example below will support KaTeX output from pandoc.
+If `DISABLE_SANITIZER` is false, Gitea supports customizing the sanitization policy for rendered HTML. The example below will support KaTeX output from pandoc.
 
 ```ini
 [markup.sanitizer.TeX]

docs/content/doc/advanced/config-cheat-sheet.zh-cn.md

@@ -318,6 +318,33 @@ IS_INPUT_FILE = false
 - FILE_EXTENSIONS: 关联的文档的扩展名，多个扩展名用都好分隔。
 - RENDER_COMMAND: 工具的命令行命令及参数。
 - IS_INPUT_FILE: 输入方式是最后一个参数为文件路径还是从标准输入读取。
+- DISABLE_SANITIZER: **false** 如果为 true 则不过滤 HTML 标签和属性。除非你知道这意味着什么，否则不要设置为 true。
+
+以下两个环境变量将会被传递给渲染命令：
+
+- `GITEA_PREFIX_SRC`：包含当前的`src`路径的URL前缀，可以被用于链接的前缀。
+- `GITEA_PREFIX_RAW`：包含当前的`raw`路径的URL前缀，可以被用于图片的前缀。
+
+如果 `DISABLE_SANITIZER` 为 false，则 Gitea 支持自定义渲染 HTML 的净化策略。以下例子将用 pandoc 支持 KaTeX 输出。
+
+```ini
+[markup.sanitizer.TeX]
+; Pandoc renders TeX segments as <span>s with the "math" class, optionally
+; with "inline" or "display" classes depending on context.
+ELEMENT = span
+ALLOW_ATTR = class
+REGEXP = ^\s*((math(\s+|$)|inline(\s+|$)|display(\s+|$)))+
+ALLOW_DATA_URI_IMAGES = true
+```
+
+- `ELEMENT`: 将要被应用到该策略的 HTML 元素，不能为空。
+- `ALLOW_ATTR`: 将要被应用到该策略的属性，不能为空。
+- `REGEXP`: 正则表达式，用来匹配属性的内容。如果为空，则跟属性内容无关。
+- `ALLOW_DATA_URI_IMAGES`: **false** 允许 data uri 图片 (`<img src="data:image/png;base64,..."/>`)。
+
+多个净化规则可以被同时定义，只要section名称最后一位不重复即可。如： `[markup.sanitizer.TeX-2]`。
+为了针对一种渲染类型进行一个特殊的净化策略，必须使用形如 `[markup.sanitizer.asciidoc.rule-1]` 的方式来命名 seciton。
+如果此规则没有匹配到任何渲染类型，它将会被应用到所有的渲染类型。
 
 ## Time (`time`)
 

docs/content/doc/developers/guidelines-backend.md

@@ -42,7 +42,7 @@ To maintain understandable code and avoid circular dependencies it is important
   - `modules/setting`: Store all system configurations read from ini files and has been referenced by everywhere. But they should be used as function parameters when possible.
   - `modules/git`: Package to interactive with `Git` command line or Gogit package.
 - `public`: Compiled frontend files (javascript, images, css, etc.)
-- `routers`: Handling of server requests. As it uses other Gitea packages to serve the request, other packages (models, modules or services) shall not depend on routers.
+- `routers`: Handling of server requests. As it uses other Gitea packages to serve the request, other packages (models, modules or services) must not depend on routers.
   - `routers/api` Contains routers for `/api/v1` aims to handle RESTful API requests. 
   - `routers/install` Could only respond when system is in INSTALL mode (INSTALL_LOCK=false). 
   - `routers/private` will only be invoked by internal sub commands, especially `serv` and `hooks`. 
@@ -106,10 +106,20 @@ i.e. `servcies/user`, `models/repository`.
 Since there are some packages which use the same package name, it is possible that you find packages like `modules/user`, `models/user`, and `services/user`. When these packages are imported in one Go file, it's difficult to know which package we are using and if it's a variable name or an import name. So, we always recommend to use import aliases. To differ from package variables which are commonly in camelCase, just use **snake_case** for import aliases.
 i.e. `import user_service "code.gitea.io/gitea/services/user"`
 
+### Important Gotchas
+
+- Never write `x.Update(exemplar)` without an explicit `WHERE` clause:
+  - This will cause all rows in the table to be updated with the non-zero values of the exemplar - including IDs.
+  - You should usually write `x.ID(id).Update(exemplar)`.
+- If during a migration you are inserting into a table using `x.Insert(exemplar)` where the ID is preset:
+  - You will need to ``SET IDENTITY_INSERT `table` ON`` for the MSSQL variant (the migration will fail otherwise)
+  - However, you will also need to update the id sequence for postgres - the migration will silently pass here but later insertions will fail:
+    ``SELECT setval('table_name_id_seq', COALESCE((SELECT MAX(id)+1 FROM `table_name`), 1), false)``
+
 ### Future Tasks
 
 Currently, we are creating some refactors to do the following things:
 
 - Correct that codes which doesn't follow the rules.
 - There are too many files in `models`, so we are moving some of them into a sub package `models/xxx`.
-- Some `modules` sub packages should be moved to `services` because they depends on `models`.
+- Some `modules` sub packages should be moved to `services` because they depend on `models`.