Files
gf/container/gpool/gpool.go
John Guo 5e72b03b0a feature/v2.3.0 (#2296)
* up

* rename function names for package gtcp/gudp; add proxy example for gtcp.Server (#2295)

* fix  router supported for handler of package ghttp; fix json tag name issue when it contains  for package goai

* add proxy example for http server

* rename function names for package gtcp/gudp; add proxy example for gtcp.Server

* move TX from struct to interface for package gdb (#2247)

* move TX from struct to interface for package gdb

* i updates

* up

* up

* fix comment

Co-authored-by: houseme <housemecn@gmail.com>

* move `go-redis` implements `Adapter` from package `gredis` to `contrib/nosql/redis`; add redis string operation functions for package `gredis` (#2240)

* unify configuration pattern of  for package gdb

* version updates

* improve implements `internal/rwmutex` and `internal/mutex`; add `TablesFields` cache implements in `gdb.Core` instead of `contrib/drivers`; add `ClearTableFields` and `ClearCache` functions for `gdb.Core` (#2128)

* add ClearTableFiels/ClearCache for Core of package gdb

* improve TableFields for contrib/drivers

* fix UT case for contrib/drivers/clickhouse

* remove unecessary attribute state for internal/rwmutex and internal/mutex

* add ClearTableFieldsAll/ClearCacheAll for gdb.Core

* improve clickhouse driver

* improve clickhouse driver

* fix ut

* feat: improve import

Co-authored-by: daguang <daguang830@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>

* refract builtin rules management mechanism, add `eq/not-eq/gt/gte/lt/lte/before/before-equal/after/after-equal/array/not-regex` rules for for package `gvalid` (#2133)

* refract builtin rules management for package gvalid

* refract builtin rules management for package gvalid

* refract builtin rules management for package gvalid

* add valiation rules  and  implements for package gvalid

* UT cases update for package gvalid

* improve error message of fields validation for package gvalid

* up

* add more validation rules for package gvalid

* add validation rule foreach for package gvalid (#2136)

* add ToSQL/CatchSQL funcions for package gdb (#2137)

* add ToSQL/CatchSQL funcions for package gdb

* Update gdb_core_underlying.go

* fix ci

Co-authored-by: houseme <housemecn@gmail.com>

* add redis interface for package gredis

* up

* remove `FilteredLink` function for DB and all driver implements and improve details for package gdb (#2142)

* fix: pgsql DoExec Transaction checks (#2101)

Co-authored-by: John Guo <john@johng.cn>

* improve package gdb

* up

* up

* up

* up

* up

* add DriverWrapper and DriverWarapperDB for package gdb

* add DriverWrapper and DriverWarapperDB for package gdb

* up

Co-authored-by: HaiLaz <739476267@qq.com>

* add new database driver `dm`

* add drivers dm

* upd go version

* add gf ci yaml

Co-authored-by: Xu <zhenghao.xu>

* move go-redis implements from package gredis to contrib/nosql/redis; add redis string operation functions for package gredis

* improve `contrib/drivers/dm` (#2144)

* improve contrib/drivers/dm

* format TODO list info

* 1) add config.Name is required
2) The upper layer no longer needs to specify the schema
3) Adjust unit tests

Co-authored-by: Xu <zhenghao.xu>
Co-authored-by: houseme <housemecn@gmail.com>

* move redis adapter related ut case from package gcache/gsession to package contrib/nosql/redis

* up

* up

* up

* up

* up

* improve comment

* add implements of `gcfg.Adapter` using kubernetes configmap (#2145)

* remove Logger from kubecm.Client

* README updates for package kubecm

* error message update for package gredis

* comment update for package gdb

* Feature/v2.2.0 gredis (#2155)

* improve package gredis (#2162)

* improve package gredis

* Update gredis_redis_group_list.go

* fix

* up

Co-authored-by: houseme <housemecn@gmail.com>

* up

* up

* up

* up

* up

* up

* add func Test_GroupScript_Eval

* ut cases for group string

* UT cases update for group script

* mv redis operation implements to contrib/nosql/redis from package gredis

* test: add redis group list unit test (#2248)

* test: add redis group list unit test

* improve comment

* test: fix redis group list unit test

Co-authored-by: houseme <housemecn@gmail.com>

* up

* add func Test_GroupGeneric_Copy, Test_GroupGeneric_Exists,Test_GroupGeneric_Type,Test_GroupGeneric_Unlink,Test_GroupGeneric_Rename,Test_GroupGeneric_Move,Test_GroupGeneric_Del

* add Redis GroupGeneric UnitTest (#2253)

add func Test_GroupGeneric_RandomKey,Test_GroupGeneric_DBSize,Test_GroupGeneric_Keys,Test_GroupGeneric_FlushDB,Test_GroupGeneric_FlushAll,Test_GroupGeneric_Expire,Test_GroupGeneric_ExpireAt

* hash test case completed (#2260)

Co-authored-by: junler <sunjun@bookan.com>

* add Redis GroupGeneric Unit Test part2 (#2258)

* up

* ci updates

* ci updates

* up

* Feature/contrib redis fsprouts (#2274)

* Feature/contrib redis starck (#2275)

* up

* up

* fix `/*` router supported for handler of package ghttp; fix json tag name issue when it contains `,` for package goai; add proxy example for http server (#2294)

* fix  router supported for handler of package ghttp; fix json tag name issue when it contains  for package goai

* add proxy example for http server

* fix: update szenius/set-timezone@v1.1 (#2293)

* add Tag* functions to retreive most commonly used tag value from struct field for package gstructs; use description tag as default value if brief is empty for gcmd.Argument (#2299)

* fix cache issue in Count/Value functions for gdb.Model (#2300)

* add Tag* functions to retreive most commonly used tag value from struct field for package gstructs; use description tag as default value if brief is empty for gcmd.Argument

* fix cache issue in Count/Value functions for gdb.Model

* add more ut case for package gdb

* version updates

* add minus of `start` parameter support for `gstr.Substr`, like the `substr` function in `PHP` (#2297)

* Make the substr like the substr in PHP

Make the substr like the substr in PHP

* Update gstr_z_unit_test.go

* Update gstr_z_unit_test.go

* Make the SubStrRune like the mb_substr in PHP

Make the SubStrRune like the mb_substr in PHP

* Update gstr_z_unit_test.go

* Update gstr_z_unit_test.go

* Update gins_z_unit_view_test.go

* Update gview_z_unit_test.go

* add ut cases for package gcode (#2307)

* add ut cases for package gerror (#2304)

* add ut cases for package gerror

* add ut cases for package gerror

* add ut cases for package gtime (#2303)

* add ut cases for package gtime

* add ut cases for package gtime

* add ut cases for package gtime

* add ut cases for package glog (#2302)

* add ut cases for package glog

* add ut cases for package glog

* add ut cases for package glog

* add ut cases for package glog

* add ut cases for package glog

* add ut cases for package glog

* change result data type of function Count from int to int64 for package gdb (#2298)

* feat: modify model count value int64

* fix

* fix:modify int64

* fix

* feat: cmd gf prebuild suport oracle (#2312)

* add ut cases for package g (#2315)

* add ut cases for package gdebug (#2313)

* add ut cases for package gdebug

* add ut cases for package gdebug

* add ut cases for package gdebug

Co-authored-by: houseme <housemecn@gmail.com>

* add zookeeper registry support (#2284)

* add ut cases for package glog part2 (#2317)

* fix invalid UpdatedAt usage in soft deleting feature for package gdb (#2323)

* fix issue in failed installing when there's shortcut between file paths for command install (#2326)

* fix issue in failed installing when has shortcut between file paths for command install

* version updates

* template for command gf updates

* improve lru clearing for package gcache (#2327)

* add ut cases for package ghttp_middleware and ghttp_request (#2344)

* add ut cases for package ghttp_middleware

* add ut cases for package ghttp_request

* add ut cases for package ghttp_request

* add ut cases for package ghttp_response (#2352)

* add ut cases for package ghttp_response

* add ut cases for package ghttp_response

* add ut cases for package ghttp_response

* add ut cases for package ghttp_request (#2351)

* add ut cases for package ghttp_middleware

* add ut cases for package ghttp_request

* add ut cases for package ghttp_request

* add ut cases for package ghttp_request

* add ut cases for package ghttp_request - form

* add ut cases for package ghttp_request - query

* add ut cases for package ghttp_request - request

* add ut cases for package ghttp_request - router

* add ut cases for package gcache (#2341)

* gTcp Example Function:
1.NewConn 2.NewConnTLS 3.NewConnKeyCrt

* gTcp Example Function:
1.Send

* add example function ExampleConn_Recv and ExampleConn_RecvWithTimeout

* add example function
1. ExampleConn_SendWithTimeout
2. ExampleConn_RecvLine
3. ExampleConn_RecvTill

* add example function
1. ExampleConn_SendRecv
2. ExampleConn_SendRecvWithTimeout
3. ExampleConn_SetDeadline
4. ExampleConn_SetReceiveBufferWait

* add gtcp test function
1. Test_Package_Option_HeadSize4
2. Test_Package_Option_Error

* add gtcp example function
1. ExampleGetFreePorts
2. ExampleSend
3. ExampleSendRecv
4. ExampleSendWithTimeout
5. ExampleSendRecvWithTimeout
6. ExampleMustGetFreePort

* add gtcp example function
1. ExampleSendPkg
2. ExampleSendRecvPkg
3. ExampleSendPkgWithTimeout
4. ExampleSendRecvPkgWithTimeout

* add gtcp test function
1. Test_Pool_Send
2. Test_Pool_Recv
3. Test_Pool_RecvLine
4. Test_Pool_RecvTill
5. Test_Pool_RecvWithTimeout
6. Test_Pool_SendWithTimeout
7. Test_Pool_SendRecvWithTimeout

* fix

* add gtcp example function
1. ExampleGetServer
2. ExampleSetAddress
3. ExampleSetHandler
4. ExampleRun_NilHandle

* exec CI

* exec CI

* exec CI

* modify test server address

* modify and exec CI

* modify and exec CI

* modify and exec CI

* modify and exec CI

* modify and exec CI

* modify and exec CI

* add example funcion ExampleConn_Recv_Once and fix

* fix

* add some error case in example function

* add some error case in example function

* 1.add example function ExampleNewServerKeyCrt
2.add function SendRecvPkgWithTimeout unit test

* add function Test_Server_NewServerKeyCrt unit test

* revert

* add function Test_Package_Timeout, Test_Package_Option_HeadSize3, Test_Conn_RecvPkgError unit test

* fix

* add example function
1.ExampleClient_Clone
2.ExampleLoadKeyCrt

* add example function
1.ExampleNewNetConnKeyCrt

* fix

* add example function
1.ExampleClient_DeleteBytes
2.ExampleClient_HeadBytes
3.ExampleClient_PatchBytes
4.ExampleClient_ConnectBytes
5.ExampleClient_OptionsBytes
6.ExampleClient_TraceBytes
7.ExampleClient_PutBytes

* add example function
1.ExampleClient_Prefix
2.ExampleClient_Retry
3.ExampleClient_RedirectLimit

* add example function
1.ExampleClient_SetBrowserMode
2.ExampleClient_SetHeader
3.ExampleClient_SetRedirectLimit

* add example function
1.ExampleClient_SetTLSKeyCrt
2.ExampleClient_SetTLSConfig
modify example funcion
1.ExampleClient_SetProxy
2.ExampleClient_Proxy

* add example function
1.ExampleClient_PutContent
2.ExampleClient_DeleteContent
3.ExampleClient_HeadContent
4.ExampleClient_PatchContent
5.ExampleClient_ConnectContent
6.ExampleClient_OptionsContent
7.ExampleClient_TraceContent
8.ExampleClient_RequestContent

* add example function
1.ExampleClient_RawRequest

* add unit function
1.TestGetFreePorts
2.TestNewConn
3.TestNewConnTLS
4.TestNewConnKeyCrt
5.TestConn_SendWithTimeout

* add unit function
1.TestConn_Send
2.TestConn_SendRecv
3.TestConn_SendRecvWithTimeout

* modify

* modify

* add example function
1.TestConn_SetReceiveBufferWait
2.TestNewNetConnKeyCrt
3.TestSend

* add example function
1.TestSendRecv
2.TestSendWithTimeout

* add unit function
1.TestMustGetFreePort
2.TestSendRecvWithTimeout
3.TestSendPkg

* add client recevied server's response content assert

* modify

* modify

* add example function
1.TestSendRecvPkg
2.TestSendPkgWithTimeout
3.TestSendRecvPkgWithTimeout

* add GetAddress() function
add unit funciton
1.TestNewServer
2.TestGetServer
3.TestServer_SetAddress
4.TestServer_SetHandler
5.TestServer_Run

* modify

* modify

* add unit funciton
1.TestLoadKeyCrt

* modify

* delete function fromHex

* add gclient dump unit test

* add example function
1.ExampleClient_Put
2.ExampleClient_Delete
3.ExampleClient_Head
4.ExampleClient_Patch
5.ExampleClient_Connect
6.ExampleClient_Options
7.ExampleClient_Trace

* add example function
1.TestClient_DoRequest

* add example function
1.ExampleClient_PutVar
2.ExampleClient_DeleteVar
3.ExampleClient_HeadVar
4.ExampleClient_PatchVar
5.ExampleClient_ConnectVar
6.ExampleClient_OptionsVar
7.ExampleClient_TraceVar

* modify

* modify

* add CustomProvider function

* modify

* add unit funciton
1.Test_NewConn
2.Test_GetFreePorts

* add unit funciton
1.Test_Server

* garray_normal_any code converage

* garray_normal_int code converage

* garray_normal_str code converage

* garray_sorted_any code converage

* garray_sorted_int code converage

* garray_sorted_str code converage

* glist code converage

* gmap, gmap_hash_any_any_map code converage

* gmap_hash_int_any_map code converage

* gmap_hash_int_any_map code converage

* gmap_hash_int_int_map code converage

* gmap_hash_int_str_map code converage

* gmap_hash_str_any_map code converage

* gmap_hash_str_int_map code converage

* gmap_hash_str_str_map code converage

* gmap_list_map code converage

* gmap_list_map code converage

* revert gf.yml

* add gtest unit test function

* add ut cases for package gcache

* add ut cases for package gcache

* add ut cases for package gcache

* add ut cases for package gcache

* add ut cases for package gcache

* modify

Co-authored-by: John Guo <john@johng.cn>

* improve ut case for package internal/rwmutex (#2364)

* fix issue when only one file was uploaded in batch receiver attribute (#2365)

* fix fixed An error occurred when only one file was uploaded in batches and add unit testing(#2092)

* fix issue uploading files for ghttp.Server

Co-authored-by: yxh <yxh1103@qq.com>

* fix issue #2334 when accessing static files with cache time (#2366)

* Solve the problem of error when accessing static files with cache time.
Error message:
2022-11-29 19:40:11.090 [ERRO] http: superfluous response.WriteHeader call from github.com/gogf/gf/v2/net/ghttp.(*ResponseWriter).Flush (ghttp_response_writer.go:58)
Stack:

Verification method:
curl 'http://127.0.0.1:8000/' -H 'If-Modified-Since: Thu, 08 Dec 2022 03:13:55 GMT' --compressed

* Solve the problem of error when accessing static files with cache time.
Error message:
2022-11-29 19:40:11.090 [ERRO] http: superfluous response.WriteHeader call from github.com/gogf/gf/v2/net/ghttp.(*ResponseWriter).Flush (ghttp_response_writer.go:58)
Stack:

Verification method:
curl 'http://127.0.0.1:8000/' -H 'If-Modified-Since: Thu, 08 Dec 2022 03:13:55 GMT' --compressed

* Solve the problem of error when accessing static files with cache time.
Error message:
2022-11-29 19:40:11.090 [ERRO] http: superfluous response.WriteHeader call from github.com/gogf/gf/v2/net/ghttp.(*ResponseWriter).Flush (ghttp_response_writer.go:58)
Stack:

Verification method:
curl 'http://127.0.0.1:8000/' -H 'If-Modified-Since: Thu, 08 Dec 2022 03:13:55 GMT' --compressed

* fix issue #2334 when accessing static files with cache time

* up

Co-authored-by: 曾洪亮 <hongliang.zeng@i-soft.com.cn>
Co-authored-by: houseme <housemecn@gmail.com>

* fix issue in cycle dumping for g.Dump (#2367)

* fix issue in cycle dumping for g.Dump

* up

* up

* up

Co-authored-by: houseme <housemecn@gmail.com>

* 由于 clickhouse 的 position的初始值为 1,导致gdb_core_utility.HasField 中对 fieldsArray 初始化出错 (#2346)

* 由于 clickhouse 的 position的初始值为 1,导致gdb_core_utility.HasField 中对 fieldsArray 初始化出错

* 修复单元测试

* 修复单元测试

* 补充单元测试

* 增加CK防御性代码

Co-authored-by: longl <longlei@dealmap.cloud>
Co-authored-by: houseme <housemecn@gmail.com>

* fix: ghttp server static path config (#2335)

Co-authored-by: daguang <daguang830@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: ftl <1139556759@qq.com>
Co-authored-by: HaiLaz <739476267@qq.com>
Co-authored-by: zhonghuaxunGM <50815786+zhonghuaxunGM@users.noreply.github.com>
Co-authored-by: huangqian <huangqian1985@qq.com>
Co-authored-by: junler <827640651@qq.com>
Co-authored-by: junler <sunjun@bookan.com>
Co-authored-by: Starccck <28645972+starccck@users.noreply.github.com>
Co-authored-by: Jinhongyu <30454170+cnjinhy@users.noreply.github.com>
Co-authored-by: YuanXin Hu <huyuanxin1999@outlook.com>
Co-authored-by: yxh <yxh1103@qq.com>
Co-authored-by: 曾洪亮 <hongliang.zeng@i-soft.com.cn>
Co-authored-by: long <48313408+qq375251855@users.noreply.github.com>
Co-authored-by: longl <longlei@dealmap.cloud>

Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: daguang <daguang830@gmail.com>
Co-authored-by: ftl <1139556759@qq.com>
Co-authored-by: HaiLaz <739476267@qq.com>
Co-authored-by: zhonghuaxunGM <50815786+zhonghuaxunGM@users.noreply.github.com>
Co-authored-by: huangqian <huangqian1985@qq.com>
Co-authored-by: junler <827640651@qq.com>
Co-authored-by: junler <sunjun@bookan.com>
Co-authored-by: Starccck <28645972+starccck@users.noreply.github.com>
Co-authored-by: Jinhongyu <30454170+cnjinhy@users.noreply.github.com>
Co-authored-by: YuanXin Hu <huyuanxin1999@outlook.com>
Co-authored-by: yxh <yxh1103@qq.com>
Co-authored-by: 曾洪亮 <hongliang.zeng@i-soft.com.cn>
Co-authored-by: long <48313408+qq375251855@users.noreply.github.com>
Co-authored-by: longl <longlei@dealmap.cloud>
2023-01-09 14:43:10 +08:00

189 lines
5.1 KiB
Go

// Copyright GoFrame Author(https://goframe.org). All Rights Reserved.
//
// This Source Code Form is subject to the terms of the MIT License.
// If a copy of the MIT was not distributed with this file,
// You can obtain one at https://github.com/gogf/gf.
// Package gpool provides object-reusable concurrent-safe pool.
package gpool
import (
"context"
"time"
"github.com/gogf/gf/v2/container/glist"
"github.com/gogf/gf/v2/container/gtype"
"github.com/gogf/gf/v2/errors/gcode"
"github.com/gogf/gf/v2/errors/gerror"
"github.com/gogf/gf/v2/os/gtime"
"github.com/gogf/gf/v2/os/gtimer"
)
// Pool is an Object-Reusable Pool.
type Pool struct {
list *glist.List // Available/idle items list.
closed *gtype.Bool // Whether the pool is closed.
TTL time.Duration // Time To Live for pool items.
NewFunc func() (interface{}, error) // Callback function to create pool item.
// ExpireFunc is the for expired items destruction.
// This function needs to be defined when the pool items
// need to perform additional destruction operations.
// Eg: net.Conn, os.File, etc.
ExpireFunc func(interface{})
}
// Pool item.
type poolItem struct {
value interface{} // Item value.
expireAt int64 // Expire timestamp in milliseconds.
}
// NewFunc Creation function for object.
type NewFunc func() (interface{}, error)
// ExpireFunc Destruction function for object.
type ExpireFunc func(interface{})
// New creates and returns a new object pool.
// To ensure execution efficiency, the expiration time cannot be modified once it is set.
//
// Note the expiration logic:
// ttl = 0 : not expired;
// ttl < 0 : immediate expired after use;
// ttl > 0 : timeout expired;
func New(ttl time.Duration, newFunc NewFunc, expireFunc ...ExpireFunc) *Pool {
r := &Pool{
list: glist.New(true),
closed: gtype.NewBool(),
TTL: ttl,
NewFunc: newFunc,
}
if len(expireFunc) > 0 {
r.ExpireFunc = expireFunc[0]
}
gtimer.AddSingleton(context.Background(), time.Second, r.checkExpireItems)
return r
}
// Put puts an item to pool.
func (p *Pool) Put(value interface{}) error {
if p.closed.Val() {
return gerror.NewCode(gcode.CodeInvalidOperation, "pool is closed")
}
item := &poolItem{
value: value,
}
if p.TTL == 0 {
item.expireAt = 0
} else {
// As for Golang version < 1.13, there's no method Milliseconds for time.Duration.
// So we need calculate the milliseconds using its nanoseconds value.
item.expireAt = gtime.TimestampMilli() + p.TTL.Nanoseconds()/1000000
}
p.list.PushBack(item)
return nil
}
// MustPut puts an item to pool, it panics if any error occurs.
func (p *Pool) MustPut(value interface{}) {
if err := p.Put(value); err != nil {
panic(err)
}
}
// Clear clears pool, which means it will remove all items from pool.
func (p *Pool) Clear() {
if p.ExpireFunc != nil {
for {
if r := p.list.PopFront(); r != nil {
p.ExpireFunc(r.(*poolItem).value)
} else {
break
}
}
} else {
p.list.RemoveAll()
}
}
// Get picks and returns an item from pool. If the pool is empty and NewFunc is defined,
// it creates and returns one from NewFunc.
func (p *Pool) Get() (interface{}, error) {
for !p.closed.Val() {
if r := p.list.PopFront(); r != nil {
f := r.(*poolItem)
if f.expireAt == 0 || f.expireAt > gtime.TimestampMilli() {
return f.value, nil
} else if p.ExpireFunc != nil {
// TODO: move expire function calling asynchronously out from `Get` operation.
p.ExpireFunc(f.value)
}
} else {
break
}
}
if p.NewFunc != nil {
return p.NewFunc()
}
return nil, gerror.NewCode(gcode.CodeInvalidOperation, "pool is empty")
}
// Size returns the count of available items of pool.
func (p *Pool) Size() int {
return p.list.Len()
}
// Close closes the pool. If `p` has ExpireFunc,
// then it automatically closes all items using this function before it's closed.
// Commonly you do not need to call this function manually.
func (p *Pool) Close() {
p.closed.Set(true)
}
// checkExpire removes expired items from pool in every second.
func (p *Pool) checkExpireItems(ctx context.Context) {
if p.closed.Val() {
// If p has ExpireFunc,
// then it must close all items using this function.
if p.ExpireFunc != nil {
for {
if r := p.list.PopFront(); r != nil {
p.ExpireFunc(r.(*poolItem).value)
} else {
break
}
}
}
gtimer.Exit()
}
// All items do not expire.
if p.TTL == 0 {
return
}
// The latest item expire timestamp in milliseconds.
var latestExpire int64 = -1
// Retrieve the current timestamp in milliseconds, it expires the items
// by comparing with this timestamp. It is not accurate comparison for
// every item expired, but high performance.
var timestampMilli = gtime.TimestampMilli()
for {
if latestExpire > timestampMilli {
break
}
if r := p.list.PopFront(); r != nil {
item := r.(*poolItem)
latestExpire = item.expireAt
// TODO improve the auto-expiration mechanism of the pool.
if item.expireAt > timestampMilli {
p.list.PushFront(item)
break
}
if p.ExpireFunc != nil {
p.ExpireFunc(item.value)
}
} else {
break
}
}
}