87 Commits

Author SHA1 Message Date
wxiaoguang
e42a1dbb6b Refactor GetRepoRawDiffForFile to avoid unnecessary pipe or goroutine (#36434) 2026-01-23 02:10:11 +00:00
wxiaoguang
3a09d7aa8d Refactor git command stdio pipe (#36422)
Most potential deadlock problems should have been fixed, and new code is
unlikely to cause new problems with the new design.

Also raise the minimum Git version required to 2.6.0 (released in 2015)
2026-01-22 06:04:26 +00:00
Lunny Xiao
f6db180a80 Fix missing repository id when migrating release attachments (#36389)
This PR fixes missed repo_id on the migration of attachments to Gitea.
It also provides a doctor check to fix the dirty data on the database.
2026-01-20 10:05:51 -08:00
wxiaoguang
72be55f7d3 Refactor git command stderr handling (#36402)
And clean up legacy fragile & incorrect logic
2026-01-18 15:10:33 -08:00
Lunny Xiao
fafd1db19e Some refactors about GetMergeBase (#36186)
Maybe fix #32018

- Use `gitrepo.GetMergeBase` method instead of other two
implementations.
- Add `FetchRemoteCommit` so that we don't need to add many `remote` to
the git repository to avoid possible git lock conflicts. A lock will
start when invoke the function, it will be invoked when cross-repository
comparing. The head repository will fetch the base repository's base
commit id. In most situations, it should lock the fork repositories so
that it should not become a bottleneck.
- Improve `GetCompareInfo` to remove unnecessarily adding remote.
- Remove unnecessary parameters of `SignMerge`.
2026-01-17 11:22:09 -08:00
Lunny Xiao
1771569300 Some refactor for repo path (#36251)
- Use `gitrepo.IsRepositoryExist` instead of `util.IsExit` or
`util.IsDir`
- Use `gitrepo.OpenRepository` instead of `git.OpenRepository`
- Use `gitrepo.DeleteRepository` instead of `util.RemoveAll`
- Use `gitrepo.RenameRepository` instead of `util.Rename`
2025-12-29 18:49:54 +00:00
Lunny Xiao
ebf9b4dc6b Use a migration test instead of a wrong test which populated the meta test repositories and fix a migration bug (#36160)
The test `TestGiteaUploadUpdateGitForPullRequest` modified the shared
meta test repositories directly, so this PR removes that test and
replaces it with an integration test that migrates a real repository
from gitea.com into a local test instance.

This PR also fixes a bug where pull-request migrations were not
correctly syncing head branches to the database.
2025-12-17 12:00:07 -08:00
Lunny Xiao
64960a18f9 Move commit related functions to gitrepo package (#35600) 2025-12-05 00:20:23 +00:00
Lunny Xiao
69f5ee970c Move some functions to gitrepo package (#35543)
Refactor Git command functions to use WithXXX methods instead of
exposing RunOpts.
This change simplifies reuse across gitrepo and improves consistency,
encapsulation, and maintainability of command options.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-10-07 17:06:51 +08:00
Lunny Xiao
198f37e33c Move updateref and removeref to gitrepo and remove unnecessary open repository (#35511)
Extracted from #35077
`UpdateRef` and `RemoveRef` will call git commands even for gogit
version.
2025-09-19 08:04:18 -07:00
Lunny Xiao
9332ff291b Move git command to git/gitcmd (#35483)
The name cmd is already used in many places and may cause conflicts, so
I chose `gitcmd` instead to minimize potential naming conflicts.
2025-09-15 23:33:12 -07:00
Lunny Xiao
37958e486a Rename pull request GetGitRefName to GetGitHeadRefName (#35093) 2025-07-16 21:33:33 +08:00
Lunny Xiao
497b83b75d Fix migration pull request title too long (#34577)
Fix #34294
2025-06-04 15:45:45 +00:00
Lunny Xiao
780e92ea99 Only git operations should update last changed of a repository (#34388)
Try to fix #32046
2025-05-11 19:18:46 +00:00
Brecht Van Lommel
a9343896f4 Option to delay conflict checking of old pull requests until page view (#27779)
`[repository.pull-request] DELAY_CHECK_FOR_INACTIVE_DAYS` is a new
setting to delay the mergeable check for pull requests that have been
inactive for the specified number of days.

This avoids potentially long delays for big repositories with many pull
requests. and reduces system load overall when there are many
repositories or pull requests.

When viewing the PR, checking will start immediately and the PR merge
box will automatically reload when complete. Accessing the PR through
the API will also start checking immediately.

The default value of `7` provides a balance between system load, and
keeping behavior similar to what it was before both for users and API
access. With `0` all conflict checking will be delayed, while `-1`
always checks immediately to restore the previous behavior.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-04-24 19:26:57 +00:00
Lunny Xiao
f6474cf2e9 Fix bug when migrating repository (#34182)
This PR fixed a bug which is a regression from #31035
2025-04-14 04:48:03 +00:00
TheFox0x7
ee3c82f874 Enable addtional linters (#34085)
enable mirror, usestdlibbars and perfsprint 
part of: https://github.com/go-gitea/gitea/issues/34083

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-04-01 10:14:01 +00:00
TheFox0x7
ae3a18e01a Remove context from git struct (#33793)
Argument is moved from struct init in command run, which lets us remove
context from struct.
2025-03-04 11:56:11 -08:00
TheFox0x7
1ec8d80fa3 refactor: decouple context from migration structs (#33399)
Use context as much as possible.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-02-07 05:37:32 +00:00
wxiaoguang
9bfa9f450d Refactor "string truncate" (#32984) 2024-12-26 11:56:03 +08:00
Zettat123
74664b08a0 Support migrating GitHub/GitLab PR draft status (#32242)
Resolve #32196
2024-10-13 22:58:13 +03:00
wxiaoguang
edbf74c418 Fix "force private" logic (#31012)
When creating a repo, the "FORCE_PRIVATE" config option should be
respected, `readonly` doesn't work for checkbox, so it should use
`disabled` attribute.
2024-05-20 00:56:45 +00:00
Chongyi Zheng
e80466f734 Resolve lint for unused parameter and unnecessary type arguments (#30750)
Resolve all cases for `unused parameter` and `unnecessary type
arguments`

Related: #30729

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2024-04-29 08:47:56 +00:00
Lunny Xiao
58ce1de994 Move migration functions to services layer (#29497) 2024-03-01 02:23:00 +00:00
Sebastian Brückner
17f170ee37 Include resource state events in Gitlab downloads (#29382)
Some specific events on Gitlab issues and merge requests are stored
separately from comments as "resource state events". With this change,
all relevant resource state events are downloaded during issue and merge
request migration, and converted to comments.

This PR also updates the template used to render comments to add support
for migrated comments of these types.

ref: https://docs.gitlab.com/ee/api/resource_state_events.html
2024-02-26 05:08:21 +01:00
Sebastian Brückner
6e5966597c Properly migrate target branch change GitLab comment (#29340)
GitLab generates "system notes" whenever an event happens within the
platform. Unlike Gitea, those events are stored and retrieved as text
comments with no semantic details. The only way to tell whether a
comment was generated in this manner is the `system` flag on the note
type.

This PR adds detection for a new specific kind of event: Changing the
target branch of a PR. When detected, it is downloaded using Gitea's
type for this event, and eventually uploaded into Gitea in the expected
format, i.e. with no text content in the comment.

This PR also updates the template used to render comments to add support
for migrated comments of this type.

ref:
11bd6dc826/app/services/system_notes/merge_requests_service.rb (L102)
2024-02-24 07:49:16 +00:00
Lunny Xiao
b79c30435f Use the database object format name but not read from git repoisitory everytime and fix possible migration wrong objectformat when migrating a sha256 repository (#29294)
Now we can get object format name from git command line or from the
database repository table. Assume the column is right, we don't need to
read from git command line every time.

This also fixed a possible bug that the object format is wrong when
migrating a sha256 repository from external.

<img width="658" alt="image"
src="https://github.com/go-gitea/gitea/assets/81045/6e9a9dcf-13bf-4267-928b-6bf2c2560423">
2024-02-24 06:55:19 +00:00
Sebastian Brückner
a70c00b80b Properly migrate automatic merge GitLab comments (#27873)
GitLab generates "system notes" whenever an event happens within the
platform. Unlike Gitea, those events are stored and retrieved as text
comments with no semantic details. The only way to tell whether a
comment was generated in this manner is the `system` flag on the note
type.

This PR adds detection for two specific kinds of events: Scheduling and
un-scheduling of automatic merges on a PR. When detected, they are
downloaded using Gitea's type for these events, and eventually uploaded
into Gitea in the expected format, i.e. with no text content in the
comment.

This PR also updates the template used to render comments to add support
for migrated comments of these two types.

ref:
11bd6dc826/app/services/system_notes/merge_requests_service.rb (L6-L17)

---------

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-02-22 07:29:03 +00:00
Lunny Xiao
5f82ead13c Simplify how git repositories are opened (#28937)
## Purpose
This is a refactor toward building an abstraction over managing git
repositories.
Afterwards, it does not matter anymore if they are stored on the local
disk or somewhere remote.

## What this PR changes
We used `git.OpenRepository` everywhere previously.
Now, we should split them into two distinct functions:

Firstly, there are temporary repositories which do not change:

```go
git.OpenRepository(ctx, diskPath)
```

Gitea managed repositories having a record in the database in the
`repository` table are moved into the new package `gitrepo`:

```go
gitrepo.OpenRepository(ctx, repo_model.Repo)
```

Why is `repo_model.Repository` the second parameter instead of file
path?
Because then we can easily adapt our repository storage strategy.
The repositories can be stored locally, however, they could just as well
be stored on a remote server.

## Further changes in other PRs
- A Git Command wrapper on package `gitrepo` could be created. i.e.
`NewCommand(ctx, repo_model.Repository, commands...)`. `git.RunOpts{Dir:
repo.RepoPath()}`, the directory should be empty before invoking this
method and it can be filled in the function only. #28940
- Remove the `RepoPath()`/`WikiPath()` functions to reduce the
possibility of mistakes.

---------

Co-authored-by: delvh <dev.lh@web.de>
2024-01-27 21:09:51 +01:00
JakobDev
1af45689f9 Only migrate the first 255 chars of a Github issue title (#28902)
Fixes #28846
2024-01-24 02:09:08 +00:00
Adam Majer
cbf923e87b Abstract hash function usage (#28138)
Refactor Hash interfaces and centralize hash function. This will allow
easier introduction of different hash function later on.

This forms the "no-op" part of the SHA256 enablement patch.
2023-12-13 21:02:00 +00:00
Nanguan Lin
49b98e45bc Fix migration panic due to an empty review comment diff (#28334)
Fix #28328 
```
func (p *PullRequestComment) GetDiffHunk() string {
	if p == nil || p.DiffHunk == nil {
		return ""
	}
	return *p.DiffHunk
}
```
This function in the package `go-github` may return an empty diff. When
it's empty, the following code will panic because it access `ss[1]`

ec1feedbf5/services/migrations/gitea_uploader.go (L861-L867)

ec1feedbf5/modules/git/diff.go (L97-L101)
2023-12-05 08:29:43 +00:00
JakobDev
76a85a4ce9 Final round of db.DefaultContext refactor (#27587)
Last part of #27065
2023-10-14 08:37:24 +00:00
JakobDev
ebe803e514 Penultimate round of db.DefaultContext refactor (#27414)
Part of #27065

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-10-11 04:24:07 +00:00
JakobDev
cc5df26680 Even more db.DefaultContext refactor (#27352)
Part of #27065

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: delvh <dev.lh@web.de>
2023-10-03 10:30:41 +00:00
JakobDev
cf0df023be More db.DefaultContext refactor (#27265)
Part of #27065

This PR touches functions used in templates. As templates are not static
typed, errors are harder to find, but I hope I catch it all. I think
some tests from other persons do not hurt.
2023-09-29 12:12:54 +00:00
JakobDev
7047df36d4 Another round of db.DefaultContext refactor (#27103)
Part of #27065

---------

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
2023-09-25 13:17:37 +00:00
JakobDev
f91dbbba98 Next round of db.DefaultContext refactor (#27089)
Part of #27065
2023-09-16 14:39:12 +00:00
Lunny Xiao
e3ed67859a Move some functions to service layer (#26969) 2023-09-08 21:09:23 +00:00
Lunny Xiao
4f32abaf94 move repository deletion to service layer (#26948)
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-09-08 04:51:15 +00:00
Lunny Xiao
b9df9fa2e2 Move createrepository from module to service layer (#26927)
Repository creation depends on many models, so moving it to service
layer is better.
2023-09-06 12:08:51 +00:00
Lunny Xiao
b167f35113 Add context parameter to some database functions (#26055)
To avoid deadlock problem, almost database related functions should be
have ctx as the first parameter.
This PR do a refactor for some of these functions.
2023-07-22 22:14:27 +08:00
wxiaoguang
f6e029e6c7 Make repo migration cancelable and fix various bugs (#24605)
Replace #12917

Close #24601
Close #12845


![image](https://github.com/go-gitea/gitea/assets/2114189/39378118-064d-40fb-8396-4579ebf33917)


![image](https://github.com/go-gitea/gitea/assets/2114189/faf37418-191c-46a6-90a8-353141e00e2d)


![image](https://github.com/go-gitea/gitea/assets/2114189/fdc8ee4d-125f-4737-9990-89bcdf9eb388)


![image](https://github.com/go-gitea/gitea/assets/2114189/9a3bd2c2-fe20-4011-81f0-990ed869d139)

---------

Co-authored-by: Yarden Shoham <git@yardenshoham.com>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-11 08:25:46 +00:00
wxiaoguang
ce9dee5a1e Introduce path Clean/Join helper functions (#23495)
Since #23493 has conflicts with latest commits, this PR is my proposal
for fixing #23371

Details are in the comments

And refactor the `modules/options` module, to make it always use
"filepath" to access local files.

Benefits:

* No need to do `util.CleanPath(strings.ReplaceAll(p, "\\", "/"))),
"/")` any more (not only one before)
* The function behaviors are clearly defined
2023-03-21 16:02:49 -04:00
Lunny Xiao
b116418f05 Use CleanPath instead of path.Clean (#23371)
As title.
2023-03-08 20:17:39 +08:00
Lauris BH
58b4143803 Add loading yaml label template files (#22976)
Extract from #11669 and enhancement to #22585 to support exclusive
scoped labels in label templates

* Move label template functionality to label module
* Fix handling of color codes
* Add Advanced label template
2023-03-02 01:44:23 +02:00
Sybren
b383652e02 Fix assignment to cm.AssigneeID when importing comments (#22528)
This is a fix for https://github.com/go-gitea/gitea/pull/22510

The code assumed that the `AssigneeID` from the comment YAML was an
`int64`, but it is actually an `int`, causing a panic. It also had no
check on whether the type cast was actually valid, so badly formatted
YAML could also cause a panic.

Both these issues have been fixed.
2023-01-19 13:24:40 -05:00
Sybren
151b1a9508 Support importing comment types (#22510)
This commit adds support for specifying comment types when importing
with `gitea restore-repo`. It makes it possible to import issue changes,
such as "title changed" or "assigned user changed".

An earlier version of this pull request was made by Matti Ranta, in
https://future.projects.blender.org/blender-migration/gitea-bf/pulls/3

There are two changes with regard to Matti's original code:

1. The comment type was an `int64` in Matti's code, and is now using a
string. This makes it possible to use `comment_type: title`, which is
more reliable and future-proof than an index into an internal list in
the Gitea Go code.

2. Matti's code also had support for including labels, but in a way that
would require knowing the database ID of the labels before the import
even starts, which is impossible. This can be solved by using label
names instead of IDs; for simplicity I I left that out of this PR.
2023-01-18 21:14:56 -05:00
Jason Song
71ca3067bc Check primary keys for all tables and drop ForeignReference (#21721)
Some dbs require that all tables have primary keys, see
- #16802
- #21086

We can add a test to keep it from being broken again.

Edit:

~Added missing primary key for `ForeignReference`~ Dropped the
`ForeignReference` table to satisfy the check, so it closes #21086.

More context can be found in comments.

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: zeripath <art27@cantab.net>
2022-12-23 19:35:43 +08:00
Lunny Xiao
0a7d3ff786 refactor some functions to support ctx as first parameter (#21878)
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: Lauris BH <lauris@nix.lv>
2022-12-03 10:48:26 +08:00