13 commits

Author	SHA1	Message	Date
Sandeep Bhat	547c173dab	Support sanitising the URL by removing extra slashes in the URL (#21333) ... Changes in this PR : Strips incoming request URL of additional slashes (/). For example an input like `https://git.data.coop//halfd/new-website.git` is translated to `https://git.data.coop/halfd/new-website.git` Fixes https://github.com/go-gitea/gitea/issues/20462 Fix #23242 --------- Co-authored-by: zeripath <art27@cantab.net> Co-authored-by: Jason Song <i@wolfogre.com> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>	2023-03-04 21:31:24 +08:00
Lunny Xiao	c53ad052d8	Refactor the setting to make unit test easier (#22405) ... Some bugs caused by less unit tests in fundamental packages. This PR refactor `setting` package so that create a unit test will be easier than before. - All `LoadFromXXX` files has been splited as two functions, one is `InitProviderFromXXX` and `LoadCommonSettings`. The first functions will only include the code to create or new a ini file. The second function will load common settings. - It also renames all functions in setting from `newXXXService` to `loadXXXSetting` or `loadXXXFrom` to make the function name less confusing. - Move `XORMLog` to `SQLLog` because it's a better name for that. Maybe we should finally move these `loadXXXSetting` into the `XXXInit` function? Any idea? --------- Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: delvh <dev.lh@web.de>	2023-02-20 00:12:01 +08:00
Lunny Xiao	bd820aa9c5	Add context cache as a request level cache (#22294) ... To avoid duplicated load of the same data in an HTTP request, we can set a context cache to do that. i.e. Some pages may load a user from a database with the same id in different areas on the same page. But the code is hidden in two different deep logic. How should we share the user? As a result of this PR, now if both entry functions accept `context.Context` as the first parameter and we just need to refactor `GetUserByID` to reuse the user from the context cache. Then it will not be loaded twice on an HTTP request. But of course, sometimes we would like to reload an object from the database, that's why `RemoveContextData` is also exposed. The core context cache is here. It defines a new context ```go type cacheContext struct { ctx context.Context data map[any]map[any]any lock sync.RWMutex } var cacheContextKey = struct{}{} func WithCacheContext(ctx context.Context) context.Context { return context.WithValue(ctx, cacheContextKey, &cacheContext{ ctx: ctx, data: make(map[any]map[any]any), }) } ``` Then you can use the below 4 methods to read/write/del the data within the same context. ```go func GetContextData(ctx context.Context, tp, key any) any func SetContextData(ctx context.Context, tp, key, value any) func RemoveContextData(ctx context.Context, tp, key any) func GetWithContextCache[T any](ctx context.Context, cacheGroupKey string, cacheTargetID any, f func() (T, error)) (T, error) ``` Then let's take a look at how `system.GetString` implement it. ```go func GetSetting(ctx context.Context, key string) (string, error) { return cache.GetWithContextCache(ctx, contextCacheKey, key, func() (string, error) { return cache.GetString(genSettingCacheKey(key), func() (string, error) { res, err := GetSettingNoCache(ctx, key) if err != nil { return "", err } return res.SettingValue, nil }) }) } ``` First, it will check if context data include the setting object with the key. If not, it will query from the global cache which may be memory or a Redis cache. If not, it will get the object from the database. In the end, if the object gets from the global cache or database, it will be set into the context cache. An object stored in the context cache will only be destroyed after the context disappeared.	2023-02-15 21:37:34 +08:00
flynnnnnnnnnn	e81ccc406b	Implement FSFE REUSE for golang files (#21840) ... Change all license headers to comply with REUSE specification. Fix #16132 Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github> Co-authored-by: John Olheiser <john.olheiser@gmail.com>	2022-11-27 18:20:29 +00:00
zeripath	c88547ce71	Add Goroutine stack inspector to admin/monitor (#19207) ... Continues on from #19202. Following the addition of pprof labels we can now more easily understand the relationship between a goroutine and the requests that spawn them. This PR takes advantage of the labels and adds a few others, then provides a mechanism for the monitoring page to query the pprof goroutine profile. The binary profile that results from this profile is immediately piped in to the google library for parsing this and then stack traces are formed for the goroutines. If the goroutine is within a context or has been created from a goroutine within a process context it will acquire the process description labels for that process. The goroutines are mapped with there associate pids and any that do not have an associated pid are placed in a group at the bottom as unbound. In this way we should be able to more easily examine goroutines that have been stuck. A manager command `gitea manager processes` is also provided that can export the processes (with or without stacktraces) to the command line. Signed-off-by: Andrew Thornton <art27@cantab.net>	2022-03-31 19:01:43 +02:00
KN4CK3R	3f280f89e7	Update HTTP status codes to modern codes (#18063) ... * 2xx/3xx/4xx/5xx -> http.Status... * http.StatusFound -> http.StatusTemporaryRedirect * http.StatusMovedPermanently -> http.StatusPermanentRedirect	2022-03-23 12:54:07 +08:00
6543	54e9ee37a7	format with gofumpt (#18184) ... * gofumpt -w -l . * gofumpt -w -l -extra . * Add linter * manual fix * change make fmt	2022-01-20 18:46:10 +01:00
wxiaoguang	5bf8d5445e	Refactor Router Logger (#17308) ... Make router logger more friendly, show the related function name/file/line. [BREAKING] This PR substantially changes the logging format of the router logger. If you use this logging for monitoring e.g. fail2ban you will need to update this to match the new format.	2022-01-20 19:41:25 +08:00
zeripath	16adaaeaa3	Instead of using routerCtx just escape the url before routing (#18086) ... A consequence of forcibly setting the RoutePath to the escaped url is that the auto routing to endpoints without terminal slashes fails (Causing #18060.) This failure raises the possibility that forcibly setting the RoutePath causes other unexpected behaviors too. Therefore, instead we should simply pre-escape the URL in the process registering handler. Then the request URL will be properly escaped for all the following calls. Fix #17938 Fix #18060 Replace #18062 Replace #17997 Signed-off-by: Andrew Thornton <art27@cantab.net>	2021-12-25 00:50:49 +08:00
zeripath	957c3fcb59	Update chi/middleware to chi/v5/middleware (#17888) ... Fix #17880 Co-authored-by: Lauris BH <lauris@nix.lv>	2021-12-02 21:58:08 +01:00
zeripath	01087e9eef	Make Requests Processes and create process hierarchy. Associate OpenRepository with context. (#17125) ... This PR registers requests with the process manager and manages hierarchy within the processes. Git repos are then associated with a context, (usually the request's context) - with sub commands using this context as their base context. Signed-off-by: Andrew Thornton <art27@cantab.net>	2021-11-30 20:06:32 +00:00
Lunny Xiao	f494776931	Use a variable but a function for IsProd because of a slight performance increment (#17368)	2021-10-20 16:37:19 +02:00
Lunny Xiao	1bfb0a24d8	Refactor routers directory (#15800) ... * refactor routers directory * move func used for web and api to common * make corsHandler a function to prohibit side efects * rm unused func Co-authored-by: 6543 <6543@obermui.de>	2021-06-09 01:33:54 +02:00