## Summary
- Port 4 test files from PgSQL driver to GaussDB driver to align test
coverage
- Add `gaussdb_z_unit_transaction_test.go` (40 tests): nested
transactions, savepoints, rollback, panic recovery, context propagation
- Add `gaussdb_z_unit_model_where_test.go` (35 tests): comprehensive
Where clause combinations (map, slice, struct, pointer, operators, nil,
empty)
- Add `gaussdb_z_unit_feature_hook_test.go` (6 tests): model hook
callbacks (Select/Insert/Update/Delete)
- Add `gaussdb_z_unit_feature_ctx_test.go` (5 tests): context
propagation, timeout, logging with context
- Remove old `Test_Model_Where` (2 sub-tests) from
`gaussdb_z_unit_model_test.go`, replaced by comprehensive version in
dedicated where test file (35 tests)
- **86 new test functions**, ~3,224 net new lines
## Test plan
- [x] `go build ./...` passes
- [x] `gofmt` and `gci` applied
- [x] No remaining `pgsql` references in new files
- [ ] Run `go test -v -run "Test_TX_" -count=1` against GaussDB instance
- [ ] Run `go test -v -run "Test_Model_Where" -count=1` against GaussDB
instance
- [ ] Run `go test -v -run "Test_Model_Hook" -count=1` against GaussDB
instance
- [ ] Run `go test -v -run "Test_Ctx" -count=1` against GaussDB instance
ref #4689
## Summary
- Add `pgsql_z_unit_feature_union_test.go`: 4 tests for Union/UnionAll
on both db and model level
- Add `pgsql_z_unit_feature_model_do_test.go`: 10 tests for DO (Data
Object) pattern - insert, batch insert, update, pointer fields, WHERE,
DAO pattern, and field prefix handling
- Enhance `pgsql_z_unit_raw_test.go`: add `Test_Raw_Where` for subquery
NOT EXISTS and field comparison using `gdb.Raw()`, adapted for PgSQL
double-quote quoting
- Add `testdata/table_with_prefix.sql` for PgSQL-compatible FieldPrefix
test
All tests adapted from MySQL driver test suite with PgSQL-specific
adjustments:
- Nullable table schema for DO partial inserts (PgSQL NOT NULL is
stricter than MySQL)
- Double-quote identifier quoting instead of backticks
- Unquoted table aliases in generated SQL
## Test plan
- [x] All 15 new tests pass locally
- [x] Full pgsql test suite (107 tests) passes with zero regressions
ref #4689
---------
Co-authored-by: John Guo <claymore1986@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
## Summary
- Port 2 test files, 1 testdata SQL, and append Test_Raw_Where from
PgSQL driver to GaussDB driver
- Add `gaussdb_z_unit_feature_union_test.go` (4 tests): Union/UnionAll
query operations
- Add `gaussdb_z_unit_feature_model_do_test.go` (10 tests): DO
struct-based CRUD operations
- Append `Test_Raw_Where` to `gaussdb_z_unit_raw_test.go` (1 test): raw
SQL in Where with subquery and column comparison
- Add `testdata/table_with_prefix.sql` for DO prefix tests
- **15 new test functions**, ~605 net new lines
## Test plan
- [x] `go build ./...` passes
- [x] `gofmt` and `gci` applied
- [x] No remaining `pgsql` references in new files
- [ ] Run full test suite against GaussDB instance
ref #4689
---------
Co-authored-by: John Guo <claymore1986@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
This pull request primarily removes submodule management targets from
the `Makefile` and makes minor updates to the project documentation in
both English and Chinese. The most important changes are grouped below.
Makefile cleanup:
* Removed the `subup` and `subsync` targets from the `Makefile`,
eliminating commands related to updating and committing submodules.
Documentation updates:
* Updated the logo alt text in both `README.MD` and `README.zh_CN.MD`
from "goframe gf logo" to "goframe logo" for clarity.
[[1]](diffhunk://#diff-01e6d9ffed056a02cae8d8a0ec5d476a64d017bf85c0d5a94bb23ca21f33f5aaL4-R4)
[[2]](diffhunk://#diff-c93759cb9a9500f20e551c741eb167fc72825fd638d36121357feb8253ce6ac1L4-R4)
* Revised a description in `README.zh_CN.MD` to clarify the framework's
purpose, changing “一个强大的框架” to “一款强大的框架”.
* Simplified the license description in `README.zh_CN.MD` to state
"100%开源和免费".
## Summary
- Port MySQL test coverage for Builder, Subquery, Join, and Struct
features to PgSQL driver
- Add 4 new test files with 23 test functions covering builder patterns,
subquery WHERE/HAVING/Model, all JOIN types, and struct scanning
- PgSQL dialect adaptations: double-quoted identifiers, GROUP BY with
HAVING, letter-prefixed table names, int64 id assertions, removed `Uid`
field
## Test plan
- [x] Builder tests pass: `go test -v -run
"Test_Model_Builder|Test_Safe_Builder" -count=1`
- [x] Subquery tests pass: `go test -v -run "Test_Model_SubQuery"
-count=1`
- [x] Join tests pass: `go test -v -run
"Test_Model_.*Join.*|Test_Model_FieldsPrefix" -count=1`
- [x] Struct tests pass: `go test -v -run
"Test_Model_Embedded|Test_Struct|Test_Structs|Test_Model_Scan|Test_Scan_Auto"
-count=1`
- [x] Full PgSQL test suite: 113/113 PASS
ref #4689
## Summary
- Port 4 test files from PgSQL driver to GaussDB driver
- Add `gaussdb_z_unit_feature_model_builder_test.go` (2 tests): SQL
builder with raw expressions and safe mode
- Add `gaussdb_z_unit_feature_model_subquery_test.go` (3 tests):
subquery in Select/Where/Having
- Add `gaussdb_z_unit_feature_model_join_test.go` (7 tests):
LeftJoin/RightJoin/InnerJoin with various conditions
- Add `gaussdb_z_unit_feature_model_struct_test.go` (11 tests):
struct-based insert/update/scan with tag mapping
- **23 new test functions**, ~861 net new lines
## Test plan
- [x] `go build ./...` passes
- [x] `gofmt` and `gci` applied
- [x] No remaining `pgsql` references in new files
- [ ] Run full test suite against GaussDB instance
ref #4689
This pull request upgrades the GoFrame framework and all related
dependencies from version `v2.9.8` (and similar) to `v2.10.0` across the
codebase. It also refactors the `.make_version.sh` script to improve
cross-platform compatibility when editing files, and ensures
documentation reflects the new version. These changes help keep the
project up-to-date and simplify version management.
**Dependency upgrades:**
* Updated all `go.mod` files in the main repo and contrib modules to
require `github.com/gogf/gf/v2 v2.10.0` (replacing `v2.9.8` and similar)
for consistency and latest features/bugfixes.
[[1]](diffhunk://#diff-ee0abb9c50b9f91f424349123e31b7b1ba1e1e4f7497250422696c5bda2e74ceL6-R12)
[[2]](diffhunk://#diff-cef597d401b6dad225f9e2e431bdde7e53cb60bdf287624cef38a6a7bb9ae7a3L7-R7)
[[3]](diffhunk://#diff-970f7eacff9cd97a0d8a00d59ea8041eedaa21c7544c6669aaa58ca692c6b274L6-R6)
[[4]](diffhunk://#diff-c23d0ca80cd6588b7df84de8ef84713f0ce0555ba05d2d9e7f5d1e0324b1ed3aL6-R6)
[[5]](diffhunk://#diff-aa230a2b1198e6ef8afeb7f48335eb2e2f51d87d918d63c4d891fea612d18ff0L6-R6)
[[6]](diffhunk://#diff-86c2390edbede20803cd862908fe95e7207f7dbabd5089ddd4838e1f26e7fecaL6-R6)
[[7]](diffhunk://#diff-5e1af33d38ced461fc0e13981d7051e125876d1692efc3aa9cb4b7faa4c18addL7-R7)
[[8]](diffhunk://#diff-8c6247829130f219981483ccf25af699a63de99afedeb0dd5c1b7bd8ff0919bdL9-R9)
[[9]](diffhunk://#diff-accbd2d37d45e51db3fcb0468043b1e1fd53eeac9e3d3558467ef24444188d2fL7-R7)
[[10]](diffhunk://#diff-15fac9b8e76d2782594c91da72f6a6f42fc18e359c3be35bf6564ac3ca09f700L6-R7)
[[11]](diffhunk://#diff-8e1a76afd564b6073aac7b02ca59f296ae45a24da3dc4d5c40f18169f48ceba1L6-R6)
[[12]](diffhunk://#diff-00a9db26966c21305c72e8f659628dffaff0d6e9dc98a751406d2141d51a5d90L7-R7)
[[13]](diffhunk://#diff-2cbf2f66d5cb77d9f4d00e4c0ce45055620fff50c941a588da31729f09a81f1bL6-R7)
[[14]](diffhunk://#diff-20a21d07addeea398c4adb76d077875894a73b4b5b181b9df1fafe497d3fc843L6-R6)
[[15]](diffhunk://#diff-909670f1c29b0bba24faf1420504b9eacdff124c4cbbec1ddec5de60653ad007L6-R6)
[[16]](diffhunk://#diff-8eef5f0c081743f8002e0faba686e838b323cb53b749706ea42e0440aaa793f1L7-R7)
[[17]](diffhunk://#diff-82345842a29e8eaffa4f51aab96fa2aa78597e6639fe4b0ece797bc60edacea8L6-R6)
**Script improvements:**
* Refactored `.make_version.sh` to use a new `sed_inplace` function for
in-place file editing, improving cross-platform support (Linux/macOS)
and removing reliance on a global variable for the sed command.
* Updated `.make_version.sh` to use `sed_inplace` consistently for
version replacement and dependency cleanup steps, ensuring robust file
modification regardless of OS.
[[1]](diffhunk://#diff-546db9206ba1b7973e6187a1025b3904a0b08681d40d0ee4767082040fd0f661L46-R47)
[[2]](diffhunk://#diff-546db9206ba1b7973e6187a1025b3904a0b08681d40d0ee4767082040fd0f661L84-R97)
* Added a step in `.make_version.sh` to insert local development replace
directives for Go modules, streamlining local testing and development.
**Documentation updates:**
* Updated contributor badge version in `README.MD` and `README.zh_CN.MD`
to reflect the new GoFrame version (`v2.10.0`).
[[1]](diffhunk://#diff-01e6d9ffed056a02cae8d8a0ec5d476a64d017bf85c0d5a94bb23ca21f33f5aaL48-R48)
[[2]](diffhunk://#diff-c93759cb9a9500f20e551c741eb167fc72825fd638d36121357feb8253ce6ac1L48-R48)
## Summary
- Fix `gf gen service` incorrectly handling versioned imports (e.g.,
`github.com/minio/minio-go/v7` → `github.com/minio/minio-go`)
- The root cause was faulty package name inference from import paths -
Go allows package names to differ from directory names
- Solution: Keep all non-anonymous imports and let gofmt clean up unused
ones
## Changes
- Simplified `calculateImportedItems` function in
`genservice_calculate.go`
- Added test case for versioned imports and aliased imports
## Test plan
- [x] All existing genservice tests pass (`Test_Gen_Service_Default`,
`Test_Issue3328`, `Test_Issue3835`)
- [x] New test `Test_Issue4242` verifies both versioned imports and
aliased imports are preserved
- [x] Verified generated files match expected output exactly
Closes#4242
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
## Summary
- Add wildcard pattern support (`*` and `?`) for `tables` configuration
- Fix `tablesEx` wildcard to use exact match (`^$`) for consistency
- Add warning when exact table name does not exist
- Add unit tests and integration tests for MySQL and PostgreSQL
## Changes
| Configuration | Before | After |
|---------------|--------|-------|
| `tables: "user_*"` | Not supported | Matches tables starting with
"user_" |
| `tables: "*"` | Not supported | Matches all tables |
| `tablesEx: "user_*"` | Partial match | Exact match (consistent with
tables) |
## Features
- `*` matches any characters (e.g., `user_*` matches `user_info`,
`user_log`)
- `?` matches single character (e.g., `user_???` matches `user_log` but
not `user_info`)
- Mixed patterns and exact names supported (e.g., `tables:
"user_*,config"`)
- Non-existent exact table names are skipped with warning message
## Test plan
- [x] Unit tests for `containsWildcard`, `patternToRegex`,
`filterTablesByPatterns` (11 cases)
- [x] Integration tests for MySQL (5 cases)
- [x] Integration tests for PostgreSQL (1 case with tables + tablesEx)
- [x] Standard SQL syntax for cross-database compatibility
Closes#4629
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
## Summary
- Fix `gf env` and `gf build --dumpEnv` command failing when `go env`
outputs warning messages
- When `go env` outputs warnings (e.g., invalid characters in
environment variables), it returns non-zero exit code but still provides
valid output
- The original code would fail in this case
## Changes
- Only fail when `go env` returns empty output, allow non-zero exit code
with valid output
- Skip lines that don't match `key=value` format instead of failing with
Fatal error
- Add debug log for skipped lines to help troubleshooting
- Add unit tests for env command
## Related Issue
Fixes#4469
## Test plan
- [x] `gf env` command works correctly even when `go env` outputs
warnings
- [x] `gf build --dumpEnv` works correctly
- [x] Added unit tests pass
## Summary
- Fix `gf gen enums` output file created at wrong location when using
relative path
- Output was incorrectly relative to source directory instead of current
working directory
- Add `defer gfile.Chdir(originPwd)` to restore original working
directory
## Root Cause
The code calls `gfile.Chdir(realPath)` to change to source directory
before `gfile.PutContents(in.Path, ...)`, causing relative output path
to be resolved relative to source directory.
## Solution
- Convert output path to absolute using `gfile.Abs()` before `Chdir`
- Restore original working directory with `defer` (following `genpb.go`
pattern)
## Test Cases
- `Test_Gen_Enums_Issue4387_RelativePath` - standard project with
relative path
- `Test_Gen_Enums_AbsolutePath` - absolute path (should work as before)
- `Test_Gen_Enums_Issue4387_Monorepo` - monorepo mode (`cd app/xxx && gf
gen enums`)
Closes#4387
## Summary
- Fix overlapping shardingPattern matching issue where shorter patterns
incorrectly match tables meant for longer patterns
- Sort shardingPattern by length descending so longer (more specific)
patterns are matched first
- Add break after successful pattern match to prevent tables from
matching multiple patterns
## Problem
When `shardingPattern` contains overlapping prefixes like `["a_?",
"a_b_?", "a_c_?"]`:
- Tables `a_b_1`, `a_b_2` should match `a_b_?` and generate `a_b.go`
- Tables `a_c_1`, `a_c_2` should match `a_c_?` and generate `a_c.go`
- Tables `a_1`, `a_2` should match `a_?` and generate `a.go`
But without this fix, `a_?` (converted to regex `a_(.+)`) would match
`a_b_1` first, causing `a_b_?` and `a_c_?` patterns to fail to generate
their respective dao files.
## Solution
1. Sort `shardingPattern` by length descending before matching
2. Add `break` after a table matches a pattern to prevent multiple
matches
## Test plan
- [x] Added integration test `Test_Gen_Dao_Sharding_Overlapping` with
overlapping patterns
- [x] Added SQL test data file `sharding_overlapping.sql`
- [x] Verified 3 separate dao files are generated: `a.go`, `a_b.go`,
`a_c.go`
Fixes#4603
## Summary
- Reverts the behavior change introduced in PR #4587 (commit caea7ea4b)
- Restores v2.9.7 priority behavior:
- `GetWithEnv`: config file > environment variable > default value
- `GetWithCmd`: config file > command line option > default value
## Related Issue
Closes#4074
## Changes
- `os/gcfg/gcfg.go`: Restore original logic that checks config file
first, then falls back to env/cmd
- `os/gcfg/gcfg_z_example_test.go`: Restore original example test
expectations
## Summary
- Add 2 new integration test cases for `gf gen service` command
- `Test_Gen_Service_CamelCase`: tests `DstFileNameCase: "Camel"` option
to generate service files with CamelCase naming
- `Test_Gen_Service_PackagesFilter`: tests `Packages` filter option to
generate service files only for specified packages
## Test Plan
- [x] Run `go test -v -run "Test_Gen_Service" ./...` - all 5 tests pass
(3 existing + 2 new)
## Summary
- Add 2 new integration test cases for `gf gen pb` command
- `TestGenPb_MultipleTags`: tests multiple validation tags (v:required,
v:#Id > 0, v:email) and dc tags
- `TestGenPb_NestedMessage`: tests nested message structures with
various tag types
## Test Data
- Add `testdata/genpb/multiple_tags.proto` - proto file with multiple
tag annotations
- Add `testdata/genpb/nested_message.proto` - proto file with nested
message structures
## Test Plan
- [x] Run `go test -v -run "TestGenPb" ./...` - all 4 tests pass (2
existing + 2 new)
## Summary
- Enhance unit tests for the `fix` command's `doFixV25Content` function
- 5 new test cases added (total: 6)
## New Test Cases
| Test | Description |
|------|-------------|
| Test_Fix_doFixV25Content_WithReplacement | Verify actual replacement
is made |
| Test_Fix_doFixV25Content_NoMatch | Handle content without patterns |
| Test_Fix_doFixV25Content_MultipleMatches | Handle multiple occurrences
|
| Test_Fix_doFixV25Content_EmptyContent | Handle empty content |
| Test_Fix_doFixV25Content_ComplexPath | Handle complex URL paths |
## Test plan
- [x] All 6 tests pass locally
- [x] Only added new test cases to existing test file
- [x] No modifications to non-test code
## Summary
- Add comprehensive unit tests for the `genenums` package which handles
enum parsing and JSON export
- 13 new test cases covering core functionality
## Test Coverage
| Function | Tests | Description |
|----------|-------|-------------|
| `NewEnumsParser` | 2 | Parser initialization |
| `Export` | 7 | JSON export with various types |
| `ParsePackages` | 2 | Integration with Go packages |
| `EnumItem` | 1 | Data structure |
| `getStandardPackages` | 1 | Standard library detection |
## Test plan
- [x] All 13 tests pass locally
- [x] No modifications to existing code
- [x] New test file only: `genenums_z_unit_test.go`
This pull request improves YAML support for i18n translation files and
refactors content type detection and loading logic in the `gjson`
package. The main changes include more robust detection of YAML, TOML,
INI, and Properties formats, refactoring of content type handling, and
the addition of new tests to ensure correct parsing of YAML-based i18n
resources.
### Improved content type detection and loading
* Refactored content type detection logic in `gjson` to use dedicated
functions for XML, YAML, TOML, INI, and Properties formats, making the
detection more reliable and maintainable.
* Changed the content loading mechanism in `gjson` to use specific
decode functions (`gxml.Decode`, `gyaml.Decode`, etc.) for each format
instead of converting everything to JSON first, improving accuracy and
extensibility.
* Updated type definitions and struct field comments in `gjson.go` for
clarity and consistency, including changing `ContentType` to a type
alias and improving documentation.
[[1]](diffhunk://#diff-0e4432d7e4cf171c0339e01b1842530432b986948d7f839a155543623236a03fL24-R24)
[[2]](diffhunk://#diff-0e4432d7e4cf171c0339e01b1842530432b986948d7f839a155543623236a03fL38-R71)
### i18n YAML support
* Modified i18n manager to use the new `gjson.LoadPath` method for
loading translation files, ensuring correct parsing of YAML files for
i18n.
* Added new test cases and test data for loading and verifying YAML i18n
files, including edge cases and real-world translation strings.
[[1]](diffhunk://#diff-e6eacc5abab33c149f9b39d8ebe300cf4d0abe907434605991984a5969e8707dR262-R283)
[[2]](diffhunk://#diff-1bfd438797c1f9ef18ab3cb00d23ae95202e85e2362c39c3df4f1a29c55733feR421-R430)
[[3]](diffhunk://#diff-a3ee37ff2a67c9e1ba2e1617e0f5fd63eb261ad7760a07423f703538138c2decR1-R16)
### Minor improvements
* Simplified file loading logic in `gjson.LoadPath` by removing caching
and directly reading file bytes, which streamlines the code and avoids
potential cache issues.
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
## What does this PR do?
Fixes#4156
When posting form data with file upload, if a field value contains `=`
or `&`, the value was being truncated.
### Example
```go
data := g.Map{
"file": "@file:/path/to/file.txt",
"fieldName": "aaa=1&b=2",
}
client.Post(ctx, "/upload", data)
```
**Expected**: Server receives `fieldName = "aaa=1&b=2"`
**Actual (before fix)**: Server receives `fieldName = "aaa"` (truncated)
## Root Cause Analysis
The issue was caused by three problems in the original code:
### Problem 1: Global URL encoding disable (httputils.go)
```go
// Original code - PROBLEMATIC
if urlEncode {
for k, v := range m {
if gstr.Contains(k, fileUploadingKey) || gstr.Contains(gconv.String(v), fileUploadingKey) {
urlEncode = false // Disables URL encoding for ALL values!
break
}
}
}
```
When any value contained `@file:`, URL encoding was disabled for ALL
values, causing `"aaa=1&b=2"` to remain unencoded. The `&` character was
then treated as a parameter separator.
### Problem 2: Split on all `=` characters (gclient_request.go)
```go
// Original code - PROBLEMATIC
array := strings.Split(item, "=") // Splits on ALL '=' characters
```
This caused `"fieldName=aaa=1"` to be split into `["fieldName", "aaa",
"1"]`.
### Problem 3: No URL decoding for field values
URL-encoded values were written directly to the multipart form without
decoding.
## Solution
### Fix 1: Remove global URL encoding disable
Only `@file:` prefixed values are kept unencoded for file upload
detection. Other values are properly URL-encoded.
### Fix 2: Use SplitN to limit split count
```go
array := strings.SplitN(item, "=", 2) // Only split on first '='
```
### Fix 3: Add URL decoding for field values
```go
if v, err := gurl.Decode(fieldValue); err == nil {
fieldValue = v
}
```
## Compatibility Analysis
| Scenario | Before | After | Compatible |
|----------|--------|-------|------------|
| Normal form POST (no file upload) | ✅ Works | ✅ Works | ✅ Yes |
| File upload + normal field values | ✅ Works | ✅ Works | ✅ Yes |
| File upload + field values containing `=` or `&` | ❌ Truncated | ✅
Works | ✅ Fixed |
| Field value is `@file:` (no path) | ✅ Works | ✅ Works | ✅ Yes |
| Field value starts with `@file:` but file doesn't exist | ❌ Error | ❌
Error | ✅ Yes |
| User sends pre-encoded value like `"aaa%3D1"` | ✅ Works | ✅ Works | ✅
Yes |
| Content-Type: application/json | ✅ Works | ✅ Works | ✅ Yes |
| Content-Type: application/xml | ✅ Works | ✅ Works | ✅ Yes |
### Breaking Change Assessment
**No breaking changes.** The fix only affects the file upload scenario
where field values contain special characters (`=`, `&`). Previously
this scenario was broken, now it works correctly.
### Edge Cases
1. **Literal `@file:` value**: GoFrame treats `@file:` as a special
marker for file upload. This is a framework design decision and remains
unchanged.
2. **URL decode failure**: If URL decoding fails (e.g., invalid `%XX`
sequence), the original value is preserved.
## Test Coverage
Added comprehensive tests covering:
- `Test_Issue4156` - Basic fix verification
- `Test_Issue4156_MultipleSpecialChars` - Multiple `=`, `&`, `%`, `+`,
spaces
- `Test_Issue4156_MultipleFields` - Multiple fields with special
characters
- `Test_Issue4156_NoFileUpload` - Normal POST without file upload
- `Test_Issue4156_PreEncodedValue` - Pre-encoded values like `%3D`
- `Test_Issue4156_EmptyAndSpecialValues` - Edge cases (`=` at start/end,
only special chars)
- `TestBuildParams_*` - httputil.BuildParams comprehensive tests
All tests pass, including existing `Test_Issue3748` which tests the
`@file:` marker handling.
## Files Changed
- `internal/httputil/httputils.go` - Remove global URL encoding disable,
adjust `@file:` condition
- `internal/httputil/httputils_test.go` - Add comprehensive BuildParams
tests
- `net/gclient/gclient_request.go` - Use SplitN, add URL decoding
- `net/gclient/gclient_z_unit_issue_test.go` - Add Issue 4156 test cases
## Summary
- Fix false positive cycle detection in `gutil.Dump`
- Change from global pointer tracking to path-based cycle detection
- Shared references (multiple fields pointing to same object) no longer
incorrectly marked as cycles
## Problem
When using `gutil.Dump` with structs containing fields that share the
same `reflect.Type` (e.g., multiple `int` fields), the second field's
type was incorrectly displayed as `<cycle dump 0x...>`.
Example from issue:
```go
type User struct {
Id int `params:"id"`
Name int `params:"name"`
}
fields, _ := gstructs.TagFields(&user, []string{"p", "params"})
gutil.Dump(fields) // Second field's Type shows "<cycle dump>" instead of "int"
```
## Solution
Change cycle detection from global to path-based:
- Add `defer delete()` to remove pointer from tracking set when function
returns
- Only detect true cycles (A→B→A), not shared references (A,B both point
to C)
## Benchmark Comparison
Run benchmark with:
```bash
cd util/gutil && go test -bench=Benchmark_Dump -benchmem -run=^$
```
**Before fix (master branch):**
| Benchmark | ns/op | B/op | allocs/op |
|-----------|-------|------|-----------|
| Shallow | 4071 | 5989 | 85 |
| Nested20 | 105700 | 173993 | 1952 |
| Deep50 | 422515 | 692298 | 4869 |
**After fix (this PR):**
| Benchmark | ns/op | B/op | allocs/op |
|-----------|-------|------|-----------|
| Shallow | 4049 | 5989 | 85 |
| Nested20 | 103065 | 173990 | 1952 |
| Deep50 | 469502 | 692291 | 4869 |
**Performance impact**:
- Memory allocation (B/op and allocs/op) is **identical**
- Execution time is within normal variance (±5-10%)
- The `defer delete()` operation is O(1), negligible compared to
reflection overhead
## Test plan
- [x] All existing `gutil` tests pass (68 tests)
- [x] Added `Test_Dump_Issue2902_SharedPointer` - shared pointer not
marked as cycle
- [x] Added `Test_Dump_Issue2902_SameTypeFields` - original issue
scenario
- [x] Added benchmark tests for performance tracking
- [x] Verified real cycles still detected correctly
Fixes#2902
## Summary
- Fixed a bug where `Raw()` with `Where()` and
`Count()`/`ScanAndCount()` was ignoring the Where conditions in Count
queries
- The issue was in `getFormattedSqlAndArgs()` which returned `nil` for
`conditionArgs` without calling `formatCondition()` for Raw SQL in
`SelectTypeCount` case
## Changes
- Modified `database/gdb/gdb_model_select.go` to call
`formatCondition()` for Raw SQL Count queries
- Added comprehensive test cases for MySQL and PostgreSQL drivers
- Fixed incorrect test expectation in `Test_Model_Raw`
## Test plan
- [x] Added `Test_Issue4500` with 6 edge cases covering:
- Raw SQL with WHERE + external Where condition
- Raw SQL without WHERE + external Where condition
- Raw + Where + ScanAndCount
- Raw + multiple Where conditions
- Raw SQL with no external Where (baseline)
- Verify All() still works correctly
- [x] All tests pass on PostgreSQL
Closes#4500
When underlying database drivers panic during SQL operations, the
`DoCommit` function would propagate the panic unhandled, causing Insert
operations to block indefinitely instead of returning proper errors.
This was particularly problematic with ClickHouse when using `*big.Int`
values that exceed column type limits (e.g., int128).
## Problem
The issue manifested in the following scenario:
1. User inserts data with `*big.Int` value larger than ClickHouse int128
capacity
2. ClickHouse driver panics with `"math/big: buffer too small to fit
value"`
3. Panic propagates through the call stack: `big.nat.bytes` → ClickHouse
driver → `gdb.(*Core).DoCommit`
4. Insert operation blocks indefinitely, returning neither success nor
error
## Solution
Added comprehensive panic recovery to the `DoCommit` function in
`database/gdb/gdb_core_underlying.go`:
```go
// Panic recovery to handle panics from underlying database drivers
defer func() {
if exception := recover(); exception != nil {
if err == nil {
if v, ok := exception.(error); ok && gerror.HasStack(v) {
err = v
} else {
err = gerror.WrapCodef(gcode.CodeDbOperationError,
gerror.NewCodef(gcode.CodeInternalPanic, "%+v", exception),
FormatSqlWithArgs(in.Sql, in.Args))
}
}
}
}()
```
## Benefits
- **Prevents blocking**: Insert operations now return errors instead of
hanging
- **Proper error context**: Errors include full SQL statement and
arguments for debugging
- **Graceful degradation**: Applications can handle driver panics
appropriately
- **Backward compatibility**: No breaking changes to existing
functionality
- **Universal coverage**: Protects against panics from any database
driver
## Testing
Added comprehensive tests covering:
- String panic values (e.g., "math/big: buffer too small")
- Error panic values with stack traces
- Various SQL operation types (Insert, Query, Prepare, etc.)
- Error message formatting and context preservation
All existing tests continue to pass, ensuring no regressions.
Fixes#4372.
<!-- START COPILOT CODING AGENT TIPS -->
---
💡 You can make Copilot smarter by setting up custom instructions,
customizing its development environment and configuring Model Context
Protocol (MCP) servers. Learn more [Copilot coding agent
tips](https://gh.io/copilot-coding-agent-tips) in the docs.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: hailaz <739476267@qq.com>
```golang
func main() {
adapter := gcache.NewAdapterRedis(g.Redis())
g.DB().GetCache().SetAdapter(adapter)
result, count, err := g.Model("TBL_USER").Cache(gdb.CacheOption{
Duration: 100 * time.Minute,
Name: "VIP",
}).AllAndCount(false)
g.DumpJson(result)
fmt.Println(count, err)
}
```
执行这段查询后`g.DumpJson(result)`的结果是`[
{
"COUNT(1)": 5
}
]`,但是正确结果应该是五条用户信息，查看源代码后发现先执行的count查询和后来select查询都是直接使用了`VIP`这个缓存key，在redis中实际缓存key是`SelectCache:VIP`，第二步查询select获得的是count查询的缓存，所以查询结果是错的。
因此为`Model`增加一个`PageCache`方法允许用户分别设置`count query`和`data query`的缓存参数
```golang
// PageCache sets the cache feature for pagination queries. It allows to configure
// separate cache options for count query and data query in pagination.
//
// Note that, the cache feature is disabled if the model is performing select statement
// on a transaction.
func (m *Model) PageCache(countOption CacheOption, dataOption CacheOption) *Model {
model := m.getModel()
model.pageCacheOption = []CacheOption{countOption, dataOption}
model.cacheEnabled = true
return model
}
```
然后`AllAndCount`在查询时分别给两个查询设置对应的缓存参数`ScanAndCount`同理
```golang
// AllAndCount retrieves all records and the total count of records from the model.
// If useFieldForCount is true, it will use the fields specified in the model for counting;
// otherwise, it will use a constant value of 1 for counting.
// It returns the result as a slice of records, the total count of records, and an error if any.
// The where parameter is an optional list of conditions to use when retrieving records.
//
// Example:
//
// var model Model
// var result Result
// var count int
// where := []any{"name = ?", "John"}
// result, count, err := model.AllAndCount(true)
// if err != nil {
// // Handle error.
// }
// fmt.Println(result, count)
func (m *Model) AllAndCount(useFieldForCount bool) (result Result, totalCount int, err error) {
// Clone the model for counting
countModel := m.Clone()
// If useFieldForCount is false, set the fields to a constant value of 1 for counting
if !useFieldForCount {
countModel.fields = []any{Raw("1")}
}
if len(m.pageCacheOption) > 0 {
countModel = countModel.Cache(m.pageCacheOption[0])
}
// Get the total count of records
totalCount, err = countModel.Count()
if err != nil {
return
}
// If the total count is 0, there are no records to retrieve, so return early
if totalCount == 0 {
return
}
resultModel := m.Clone()
if len(m.pageCacheOption) > 1 {
resultModel = resultModel.Cache(m.pageCacheOption[1])
}
// Retrieve all records
result, err = resultModel.doGetAll(m.GetCtx(), SelectTypeDefault, false)
return
}
```
---------
Co-authored-by: houseme <housemecn@gmail.com>
## Summary
- Fix nested map conversion data loss when using `gconv.Scan()` or
`MapToMap()`
- When converting `map[string]any` to `map[string]map[string]float64`,
the nested data was lost
- Root cause: `MapToMap` incorrectly called `Struct()` for map value
types
- Solution: Separate `reflect.Map` handling from `reflect.Struct`, use
recursive `MapToMap()` for nested maps
## Test plan
- [x] Added test case reproducing original bug (nested map conversion)
- [x] Added test cases for deep nesting (3-5 levels)
- [x] Added test case for different key types
- [x] Added test case for empty nested map
- [x] Verified struct conversion still works (no regression)
- [x] Verified no infinite recursion with timeout tests
- [x] All gconv tests pass
Closes#4542
---------
Co-authored-by: hailaz <739476267@qq.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Add configurable default endpoint and metadata support to nacos
Registry,
providing a more flexible alternative to hardcoded environment variable
reads.
- Add defaultEndpoint and defaultMetadata fields to Registry struct
- Add SetDefaultEndpoint method to override service endpoints
- Add SetDefaultMetadata method to merge extra metadata
- Update Register method to use configured defaults
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
## Summary
- Fix FieldsPrefix silently dropping fields when called before LeftJoin
- When table/alias is unknown, skip filtering and return fields directly
## Test plan
- [x] Added unit test Test_Issue4595 in pgsql driver
- [x] Test covers: FieldsPrefix before LeftJoin, Fields with prefix,
FieldsPrefix after LeftJoin
Closes#4595
The gconv.UnsafeStrToBytes function has been updated to use the Go 1.20+
safe approach, as the previous implementation could cause a panic in
certain scenarios.
For example, when an HTTP request header specifies Content-Type:
application/x-www-form-urlencoded, but the actual request body contains
JSON data, the following code attempts to detect and handle this case:
```go
if !gregex.IsMatchString(`^[\w\-\[\]]+$`, name) && len(r.PostForm) == 1 {
// It might be JSON/XML content.
if s := gstr.Trim(name + strings.Join(values, " ")); len(s) > 0 {
if s[0] == '{' && s[len(s)-1] == '}' || s[0] == '<' && s[len(s)-1] == '>' {
r.bodyContent = gconv.UnsafeStrToBytes(s)
params = ""
break
}
}
}
```
However, after this assignment, bodyContent ends up with a capacity
(cap) of 0. slice operations like [:] perform stricter validation and
will panic if the capacity is 0. This causes a panic in functions such
as:
```go
body = bytes.TrimSpace(body)
func TrimSpace(s []byte) []byte {
...
return s[start:stop] // panic here due to cap == 0
}
```
The capacity (cap) of the slice returned by directly calling this
function is unpredictable, as it depends on the adjacent memory layout.
However, within the framework, this causes issues—likely because,
starting from Go 1.22, the standard library's parseForm implementation
consistently appends a trailing zero byte after the string data in
memory.
This PR fix the problem.
------------------------------------
gconv unsafe str to bytes 改用 go1.20 后的写法，之前的写法在某些场景下会 panic
例如 http 请求头为`application/x-www-form-urlencoded`,实际的 body 为 json,
经过解析后
```go
if !gregex.IsMatchString(`^[\w\-\[\]]+$`, name) && len(r.PostForm) == 1 {
// It might be JSON/XML content.
if s := gstr.Trim(name + strings.Join(values, " ")); len(s) > 0 {
if s[0] == '{' && s[len(s)-1] == '}' || s[0] == '<' && s[len(s)-1] == '>' {
r.bodyContent = gconv.UnsafeStrToBytes(s)
params = ""
break
}
}
}
```
bodyContent的 cap 为 0，由于切片操作[:]会校验 cap 为 0，会直接 panic
```go
body = bytes.TrimSpace(body)
---
func TrimSpace(s []byte) []byte {
...
return s[start:stop] // panic
}
```
直接使用这个函数得到的 cap 会是随机的, 因为跟的内存不确定，但是在框架中有问题，估计是1.22 后标准库parseForm
的时候后面内存固定跟了个 0
该 PR 修复这个问题
Co-authored-by: liov-ola <liov@olaparty.sg>
