Commit Graph

784 Commits

Author SHA1 Message Date
df463d75bc fix(database/gdb): Resolve the cache error overwriting caused by the use of fixed cache keys in pagination queries. (#4339)
```golang
func main() {
	adapter := gcache.NewAdapterRedis(g.Redis())
	g.DB().GetCache().SetAdapter(adapter)
	result, count, err := g.Model("TBL_USER").Cache(gdb.CacheOption{
		Duration: 100 * time.Minute,
		Name:     "VIP",
	}).AllAndCount(false)
	g.DumpJson(result)
	fmt.Println(count, err)
}
```
执行这段查询后`g.DumpJson(result)`的结果是`[
    {
        "COUNT(1)": 5
    }

]`,但是正确结果应该是五条用户信息,查看源代码后发现先执行的count查询和后来select查询都是直接使用了`VIP`这个缓存key,在redis中实际缓存key是`SelectCache:VIP`,第二步查询select获得的是count查询的缓存,所以查询结果是错的。
因此为`Model`增加一个`PageCache`方法允许用户分别设置`count query`和`data query`的缓存参数
```golang
// PageCache sets the cache feature for pagination queries. It allows to configure
// separate cache options for count query and data query in pagination.
//
// Note that, the cache feature is disabled if the model is performing select statement
// on a transaction.
func (m *Model) PageCache(countOption CacheOption, dataOption CacheOption) *Model {
	model := m.getModel()
	model.pageCacheOption = []CacheOption{countOption, dataOption}
	model.cacheEnabled = true
	return model
}
```
然后`AllAndCount`在查询时分别给两个查询设置对应的缓存参数`ScanAndCount`同理
```golang

// AllAndCount retrieves all records and the total count of records from the model.
// If useFieldForCount is true, it will use the fields specified in the model for counting;
// otherwise, it will use a constant value of 1 for counting.
// It returns the result as a slice of records, the total count of records, and an error if any.
// The where parameter is an optional list of conditions to use when retrieving records.
//
// Example:
//
//	var model Model
//	var result Result
//	var count int
//	where := []any{"name = ?", "John"}
//	result, count, err := model.AllAndCount(true)
//	if err != nil {
//	    // Handle error.
//	}
//	fmt.Println(result, count)
func (m *Model) AllAndCount(useFieldForCount bool) (result Result, totalCount int, err error) {
	// Clone the model for counting
	countModel := m.Clone()

	// If useFieldForCount is false, set the fields to a constant value of 1 for counting
	if !useFieldForCount {
		countModel.fields = []any{Raw("1")}
	}
	if len(m.pageCacheOption) > 0 {
		countModel = countModel.Cache(m.pageCacheOption[0])
	}

	// Get the total count of records
	totalCount, err = countModel.Count()
	if err != nil {
		return
	}

	// If the total count is 0, there are no records to retrieve, so return early
	if totalCount == 0 {
		return
	}

	resultModel := m.Clone()
	if len(m.pageCacheOption) > 1 {
		resultModel = resultModel.Cache(m.pageCacheOption[1])
	}

	// Retrieve all records
	result, err = resultModel.doGetAll(m.GetCtx(), SelectTypeDefault, false)
	return
}
```

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-01-16 10:33:05 +08:00
cd6fd247e2 fix(‎database/gdb): fix iTableName interface detection when using WithAll with .Scan on reflect.Value objects (#4606)
fix(gdb/getTableNameFromOrmTag): 修复在使用WithAll, 并且使用.Scan传入对象的情况下,
无法识别该对象字段是否实现了iTableName的接口. 因为该情况下, 传入的object是reflect.Value.

示例如下: 

type MaterialDetail struct {
*entity.Material
SourceFile MaterialSourceFileDetail json:"source_file"
orm:"with:id=source_file_id"
}

type MaterialSourceFileDetail struct {
*entity.MaterialSourceFile
}

func (MaterialSourceFileDetail) TableName() string {
return dao.MaterialSourceFile.Table()
}

func foo(ctx context.Context) {

err = dao.Material.Ctx(ctx).WithAll().
	Where(dao.Material.Columns().MaterialId, materialId).
	Scan(&material)
}

这种情况下, 传入getTableNameFromOrmTag的object是reflect.Value, 而不是对象本身.
这会导致识别出MaterialSourceFileDetail已经实现了iTableName接口, 无法获取到正确的表名.

---------

Co-authored-by: hailaz <739476267@qq.com>
2026-01-15 21:23:07 +08:00
9dd43cd331 feat(gdb/gdb_model_lock.go): gdb support lock update skip locked (#4607)
feat(gdb/gdb_model_lock.go): GDB 支持 FOR UPDATE SKIP LOCKED 语法

---------

Co-authored-by: hailaz <739476267@qq.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-01-15 13:27:25 +08:00
3e73e2d2cc fix(database/gdb): skip field filtering when table/alias is unknown in FieldsPrefix (#4602)
## Summary
- Fix FieldsPrefix silently dropping fields when called before LeftJoin
- When table/alias is unknown, skip filtering and return fields directly

## Test plan
- [x] Added unit test Test_Issue4595 in pgsql driver
- [x] Test covers: FieldsPrefix before LeftJoin, Fields with prefix,
FieldsPrefix after LeftJoin

Closes #4595
2026-01-15 10:25:40 +08:00
7daf916032 refactor(database/gdb): simplify order and group by alias quoting (bu… (#4555)
## What this PR does

Revert the auto table prefix behavior in `Order()` and `Group()`
introduced by #4521.

  ## Why

PR #4521 attempted to resolve column ambiguity in GROUP BY/ORDER BY with
MySQL JOIN by automatically adding table prefixes to unqualified
columns. However,
  this approach has issues:

1. When using `.As()` to set table alias, it uses the original table
name instead of the alias, causing errors in PostgreSQL and other
databases
2. The framework cannot reliably determine which table the user intends
when multiple tables have the same column
  3. Adds hidden behavior that users may not expect

  ## Example of the issue (#4554)

  ```go
  db.Model("demo_a").As("a").
      LeftJoin("demo_b", "b", "a.id=b.data_id").
      Order("sort").All()

  Expected (v2.9.5):
  ORDER BY "sort"

  Actual (v2.9.6):
  ORDER BY "demo_a".sort  -- Wrong! Should use alias "a" or no prefix

  Solution

Revert to v2.9.5 behavior: Order("sort") generates ORDER BY "sort"
without auto-prefixing. Users should explicitly specify table prefix
when needed:
  Order("a.sort").

  Closes #4554
  ```

---------

Co-authored-by: hailaz <739476267@qq.com>
2025-12-26 16:43:19 +08:00
b59824e9dc feat(database/gdb): Optimize SoftTime feature (#4559)
本次PR主要针对GoFrame ORM中的软删除`SoftTime`功能进行了优化
   - 新增`SoftTimeFieldType`枚举类型,用于区分创建、更新、删除三种不同的软时间字段
   - 替代之前使用的魔数方式,提高类型安全性
   - 将原有的6个方法精简为4个方法, 合并了三个几乎相同的`GetFieldNameAndTypeFor*`方法为统一的方法


This pull request refactors and simplifies the "soft time"
(created/updated/deleted timestamp) handling logic in the database
layer, making the codebase more maintainable and extensible. The changes
consolidate multiple similar methods into general-purpose ones, improve
cache key generation, and clarify the logic for generating and applying
soft time field values and conditions.

Key changes include:

**Soft Time API Refactoring and Simplification**
- Consolidated multiple methods (`GetFieldNameAndTypeForCreate`,
`GetFieldNameAndTypeForUpdate`, `GetFieldNameAndTypeForDelete`) into a
single, parameterized method `GetFieldInfo`, reducing code duplication
and making it easier to support new soft time field types. The interface
and implementation for soft time maintenance (`iSoftTimeMaintainer`,
`softTimeMaintainer`) have been updated accordingly.
[[1]](diffhunk://#diff-6c1d606032d981a7b8aecd3a7167823f76b69407a29eb9a244175a82f59965d8L46-R66)
[[2]](diffhunk://#diff-6c1d606032d981a7b8aecd3a7167823f76b69407a29eb9a244175a82f59965d8L105-R180)
- Combined and renamed methods for generating soft time field values and
delete conditions, such as merging
`GetValueByFieldTypeForCreateOrUpdate` into `GetFieldValue`, and
`GetWhereConditionForDelete` into `GetDeleteCondition`. Related usages
throughout the codebase have been updated to use the new methods.
[[1]](diffhunk://#diff-6c1d606032d981a7b8aecd3a7167823f76b69407a29eb9a244175a82f59965d8L255-R206)
[[2]](diffhunk://#diff-97beb485550e4381182a04bbb857a25b7f4ecd4a594dff8ac884cfaae38f3046L34-R35)
[[3]](diffhunk://#diff-97beb485550e4381182a04bbb857a25b7f4ecd4a594dff8ac884cfaae38f3046L55-R55)
[[4]](diffhunk://#diff-88304ddb7791aedbd83dafb68374aecab286d1356a7f2f149a8e57ac1a7f40b4L265-R267)
[[5]](diffhunk://#diff-88304ddb7791aedbd83dafb68374aecab286d1356a7f2f149a8e57ac1a7f40b4L298-R311)
[[6]](diffhunk://#diff-d4f6e0370e049dea52f3db9a13c64e2cfb2f7ef012433186e21179149b626d0fL944-R944)

**Soft Delete Logic Improvements**
- Refactored the logic for building soft delete WHERE conditions and
generating update data, with clearer and more robust handling of field
types and prefixes. Introduced helper methods like
`buildDeleteCondition`, `GetDeleteData`, and improved error logging for
invalid field types.
[[1]](diffhunk://#diff-6c1d606032d981a7b8aecd3a7167823f76b69407a29eb9a244175a82f59965d8L287-R234)
[[2]](diffhunk://#diff-6c1d606032d981a7b8aecd3a7167823f76b69407a29eb9a244175a82f59965d8L313-R395)

**Cache Key Generation Enhancements**
- Added dedicated helper functions for generating cache keys for table
names, table fields, select queries, and soft time field/type lookups,
improving cache consistency and code readability.
[[1]](diffhunk://#diff-d57d57e6f9b342ba6fa30c4bb413e2f4f3514a8cd5ad36949eef126e5f8b7ac9R969)
[[2]](diffhunk://#diff-d57d57e6f9b342ba6fa30c4bb413e2f4f3514a8cd5ad36949eef126e5f8b7ac9R980)
[[3]](diffhunk://#diff-d57d57e6f9b342ba6fa30c4bb413e2f4f3514a8cd5ad36949eef126e5f8b7ac9R993-R1002)
[[4]](diffhunk://#diff-b1bbe5e3995261813e4e0ac6ffee8a37c236eaa2759f2bd82e211711695a70bcL790-R790)

**General Code Cleanup**
- Removed redundant code, clarified comments, and improved naming
throughout the affected files, making the code easier to follow and
maintain.
[[1]](diffhunk://#diff-6c1d606032d981a7b8aecd3a7167823f76b69407a29eb9a244175a82f59965d8L105-R180)
[[2]](diffhunk://#diff-6c1d606032d981a7b8aecd3a7167823f76b69407a29eb9a244175a82f59965d8L255-R206)

Let me know if you'd like a walkthrough of any specific part of the
refactored soft time logic!
2025-12-12 15:14:21 +08:00
852c3dda62 feat(contrib/drivers/dm&pgsql&mssql&oracle): add Replace/LastInsertId features support for dm/pgsql/mssql/oracle (#4547)
This pull request introduces significant improvements to the handling of
the `Replace` and `Save` operations for multiple database drivers,
especially for MSSQL and PostgreSQL. The changes ensure that these
operations now auto-detect primary keys when conflict columns are not
explicitly provided, improving usability and aligning behavior across
drivers. Additionally, the pull request updates related tests to reflect
these enhancements and includes some minor documentation and code
cleanup.

**Key changes:**

### Enhanced Replace/Save Logic for Database Drivers

* **MSSQL Driver:**
- `Replace` and `Save` operations now auto-detect primary keys if
`OnConflict` is not specified, using the `MERGE` statement for upsert
functionality. If no primary key is found in the data, a detailed error
is returned.
[[1]](diffhunk://#diff-87815aa559a927e2de09bd05148f9841dfc06a1b5f3ecc5e3d5fcb80323a87f8L23-R61)
[[2]](diffhunk://#diff-87815aa559a927e2de09bd05148f9841dfc06a1b5f3ecc5e3d5fcb80323a87f8L43-L59)
- Updated tests to verify that `Replace` correctly updates or inserts
records, and that missing conflict columns are properly handled.
[[1]](diffhunk://#diff-bdbde9d7d6ee14c795343767b414740c4396f4dd3e97788b1f9d4e615405a42dL141-R151)
[[2]](diffhunk://#diff-26338e93e473300b1313936eb0f6826546473793442f24715fa294b595f7a805L2661-R2707)

* **PostgreSQL Driver:**
- Similar to MSSQL, `Replace` and `Save` now auto-detect primary keys
for conflict resolution if `OnConflict` is not set, and treat `Replace`
as a `Save` operation.
- Adjusted tests to ensure `Save` and `Replace` work as expected,
including verifying data replacement and insertion.
[[1]](diffhunk://#diff-c22703c37ebb6836c332f7cd2ada570577ba4564fe39886db02f7c2d0e7a2048L93-R93)
[[2]](diffhunk://#diff-c22703c37ebb6836c332f7cd2ada570577ba4564fe39886db02f7c2d0e7a2048R102)
[[3]](diffhunk://#diff-c22703c37ebb6836c332f7cd2ada570577ba4564fe39886db02f7c2d0e7a2048L110-R130)

* **DM Driver:**
- Improved conflict detection: now checks that at least one primary key
exists in the provided data when `OnConflict` is not specified, and
provides clearer error messages.
- Refactored to use the core method for primary key detection and
removed redundant code.

### Minor Improvements and Documentation

* Added clarifying comments to `DoInsert` methods for ClickHouse, DM,
MSSQL, Oracle, and PostgreSQL drivers, specifying that the input list
must have at least one validated record.
[[1]](diffhunk://#diff-f2e003895041ed3c52b91bb8c270696adc3528d77c39d2f7137af3396267444cR19)
[[2]](diffhunk://#diff-f51b30e3f0b0f1284b905385a89992efd0de2fe9ff8c5a4062344dfab17d428eR23)
[[3]](diffhunk://#diff-87815aa559a927e2de09bd05148f9841dfc06a1b5f3ecc5e3d5fcb80323a87f8L23-R61)
[[4]](diffhunk://#diff-f61dac3fcfd5df4a3936cd8743499c8c0fc45f4f5d0f5398ed84a0cb1603202cR24)
[[5]](diffhunk://#diff-c1dfed79aaa3a432057d2bd74d270e4b4094ebcf72984f1161d4972bea009410R16-R72)
* Minor code and comment cleanups, including improved formatting and
error handling.
[[1]](diffhunk://#diff-f61dac3fcfd5df4a3936cd8743499c8c0fc45f4f5d0f5398ed84a0cb1603202cR37)
[[2]](diffhunk://#diff-f61dac3fcfd5df4a3936cd8743499c8c0fc45f4f5d0f5398ed84a0cb1603202cL96-R98)
[[3]](diffhunk://#diff-f61dac3fcfd5df4a3936cd8743499c8c0fc45f4f5d0f5398ed84a0cb1603202cL106-L116)
[[4]](diffhunk://#diff-a17b44c76aaac53d1f164a2bb9440a5531659f4355e7ccfabdadff8dc8633c09L170-R171)
[[5]](diffhunk://#diff-56189fa9ae1df51716b50d34d7fe56bfe67a330e8ac2c6b0de7b958db6817ed5R83-R98)

### Workflow and Documentation Updates

* Updated example Docker commands in the CI workflow for consistency and
clarity.
[[1]](diffhunk://#diff-a1a3cb9bdeb5541d148091d973cf266aa3b317e6415a86630e816cbe27cf8b9cL57-R57)
[[2]](diffhunk://#diff-a1a3cb9bdeb5541d148091d973cf266aa3b317e6415a86630e816cbe27cf8b9cL78-R78)
[[3]](diffhunk://#diff-a1a3cb9bdeb5541d148091d973cf266aa3b317e6415a86630e816cbe27cf8b9cL92-R92)
[[4]](diffhunk://#diff-a1a3cb9bdeb5541d148091d973cf266aa3b317e6415a86630e816cbe27cf8b9cL106-R106)
[[5]](diffhunk://#diff-a1a3cb9bdeb5541d148091d973cf266aa3b317e6415a86630e816cbe27cf8b9cL153-R153)
[[6]](diffhunk://#diff-a1a3cb9bdeb5541d148091d973cf266aa3b317e6415a86630e816cbe27cf8b9cL164-R164)
* Removed outdated note about `Replace` support from the SQLite driver
documentation.

These changes improve the consistency, reliability, and developer
experience when performing upsert operations across different database
backends.

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Lance Add <1196661499@qq.com>
2025-12-09 15:46:41 +08:00
d353bf0fbc feat(contrib/drivers/pgsql): more field types converting support (#3737)
This pull request significantly improves PostgreSQL array type handling
and conversion in the `pgsql` driver, providing more accurate type
mapping and conversion logic, especially for array types. It introduces
comprehensive documentation, refactors conversion logic to use the `pq`
package for array types, and adds extensive unit tests to ensure
correctness and error handling. Additionally, minor enhancements and
clarifications are made to upsert formatting and table field queries.

### PostgreSQL Array Type Handling and Conversion

* Refactored `CheckLocalTypeForField` and `ConvertValueForLocal` methods
in `contrib/drivers/pgsql/pgsql_convert.go` to accurately map PostgreSQL
array types (such as `_int2`, `_int4`, `_int8`, `_float4`, `_float8`,
`_bool`, `_varchar`, `_text`, `_char`, `_bpchar`, `_numeric`,
`_decimal`, `_money`, `_bytea`) to their corresponding Go types, using
the `pq` package for conversion. Added detailed documentation and
mapping tables for supported types.
[[1]](diffhunk://#diff-a3b1e68bfa29fbcfda7c703bbe875fa82e958f6c3ad942ef82193a9dd8ad67e2R46-R63)
[[2]](diffhunk://#diff-a3b1e68bfa29fbcfda7c703bbe875fa82e958f6c3ad942ef82193a9dd8ad67e2L56-R103)
[[3]](diffhunk://#diff-a3b1e68bfa29fbcfda7c703bbe875fa82e958f6c3ad942ef82193a9dd8ad67e2R112-R209)

* Added comprehensive unit tests in
`contrib/drivers/pgsql/pgsql_z_unit_convert_test.go` to verify type
mapping and conversion for all supported array types, including error
cases for invalid input.

### Utility and API Improvements

* Added a new `Bools()` method to the `gvar.Var` type in
`container/gvar/gvar_slice.go` for converting values to `[]bool`, with
corresponding unit tests in `container/gvar/gvar_z_unit_slice_test.go`.
[[1]](diffhunk://#diff-32e887e540e0170f785508d105cb794e4d54d854b53b6950973c80022973c490R11-R15)
[[2]](diffhunk://#diff-01453eca4d4b3e35d07ca105cb924c6441d0cd9df6cbcc337a89832c8d53057fR24-R41)

### SQL Formatting and Documentation

* Improved documentation and formatting in the upsert logic of
`contrib/drivers/pgsql/pgsql_format_upsert.go` to clarify the use of
`EXCLUDED` in PostgreSQL's `ON CONFLICT DO UPDATE`.
* Enhanced readability of the table field query in
`contrib/drivers/pgsql/pgsql_table_fields.go` by reformatting SQL and
clarifying field extraction.

---------

Co-authored-by: hailaz <739476267@qq.com>
Co-authored-by: houseme <housemecn@gmail.com>
2025-12-08 11:18:45 +08:00
baf30a0e99 feat(contrib/drivers/dm): add Replace/InsertIgnore support and field type/length enhancements for dm database (#4541)
This pull request introduces significant improvements to the DM database
driver, especially around insert operations, and refines documentation
and tests to reflect these changes. The main focus is enabling support
for "replace" and "insert ignore" operations using DM's `MERGE`
statement, improving type reporting for table fields, and updating
documentation for clarity and accuracy.

### DM Driver Insert Operations

* Added support for `Replace` and `InsertIgnore` operations in the DM
driver by internally mapping them to DM's `MERGE` statement. This
enables upsert and insert-ignore behavior for DM databases, improving
compatibility with other drivers.
* Implemented helper methods (`doMergeInsert`, `doInsertIgnore`, and
`getPrimaryKeys`) to generate correct `MERGE` SQL statements and
automatically detect primary keys when needed.
[[1]](diffhunk://#diff-f51b30e3f0b0f1284b905385a89992efd0de2fe9ff8c5a4062344dfab17d428eL31-R94)
[[2]](diffhunk://#diff-f51b30e3f0b0f1284b905385a89992efd0de2fe9ff8c5a4062344dfab17d428eL115-R212)
* Updated the logic for building update values and SQL generation to
ensure correct behavior for both upsert and insert-ignore cases.
[[1]](diffhunk://#diff-f51b30e3f0b0f1284b905385a89992efd0de2fe9ff8c5a4062344dfab17d428eL61-R109)
[[2]](diffhunk://#diff-f51b30e3f0b0f1284b905385a89992efd0de2fe9ff8c5a4062344dfab17d428eL89-R132)
[[3]](diffhunk://#diff-f51b30e3f0b0f1284b905385a89992efd0de2fe9ff8c5a4062344dfab17d428eL100-R144)
[[4]](diffhunk://#diff-f51b30e3f0b0f1284b905385a89992efd0de2fe9ff8c5a4062344dfab17d428eL115-R212)

### Table Field Type Reporting

* Improved the DM driver's `TableFields` method to report column types
with length/precision (e.g., `VARCHAR(128)` instead of just `VARCHAR`),
aligning with expectations and other drivers.
[[1]](diffhunk://#diff-40a365112421ae1967bd960f8acefcc91ddb8180865b78bc49cd090fbf4883daL26-R26)
[[2]](diffhunk://#diff-40a365112421ae1967bd960f8acefcc91ddb8180865b78bc49cd090fbf4883daR88-R105)
* Updated related unit tests to expect the new type format for DM table
fields.

### Documentation Updates

* Removed outdated or redundant documentation in both English and
Chinese driver README files, and clarified supported features and
limitations for DM and other drivers.
[[1]](diffhunk://#diff-d49f5bc3a34b11a6ccb82cc54675b06a7dea5f0a943ae91c4ca0d28bd5003299L1)
[[2]](diffhunk://#diff-d49f5bc3a34b11a6ccb82cc54675b06a7dea5f0a943ae91c4ca0d28bd5003299L47-R46)
[[3]](diffhunk://#diff-d49f5bc3a34b11a6ccb82cc54675b06a7dea5f0a943ae91c4ca0d28bd5003299L119-L122)
[[4]](diffhunk://#diff-05411a14e9c7ca235f7f436bfde732853aa93b364361fe80d65ac768f4e4d613L1-L126)

### Test Suite Enhancements

* Refactored and restored unit tests for DM driver insert operations,
including tests for `Save`, `Insert`, and the new `InsertIgnore`
functionality to ensure correct behavior and compatibility.
[[1]](diffhunk://#diff-2b1a59b8b2adaa1ca3074629374ab122929e4d4fbb4cc794b8e1db60ebf8d4c2L143-L245)
[[2]](diffhunk://#diff-2b1a59b8b2adaa1ca3074629374ab122929e4d4fbb4cc794b8e1db60ebf8d4c2R512-R632)
* Minor adjustments to DM test initialization for improved clarity.

### Core Insert Logic Minor Refactoring

* Minor variable renaming for clarity in the core insert logic
(`gdb_core.go`), improving code readability.
[[1]](diffhunk://#diff-b1bbe5e3995261813e4e0ac6ffee8a37c236eaa2759f2bd82e211711695a70bcL449-R452)
[[2]](diffhunk://#diff-b1bbe5e3995261813e4e0ac6ffee8a37c236eaa2759f2bd82e211711695a70bcL466-R474)

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-12-04 20:12:12 +08:00
fe8ba5e35f fix(database/gdb): Resolve column ambiguity in GROUP BY/ORDER BY with MySQL JOIN (#4521)
When using JOIN queries in MySQL with the `Group()` method, column names
in GROUP BY clauses become ambiguous if multiple tables contain columns
with the same name (commonly `id`). This results in MySQL errors like
"Column 'id' in group statement is ambiguous".

**Example Issue:**
```go
model := t.Ctx(ctx).Fields("t_inf_job.*, t_inf_job_attr.*").
    LeftJoin("t_inf_job_attr", "t_inf_job.id = t_inf_job_attr.job_id").
    Where(t.Columns().Deleted, 0)

// This would fail with "Column 'id' in group statement is ambiguous"
err = model.Group(t.Columns().Id).Scan(&jobs)
```


### **Key Changes**

1. **Modified function signature**: `Group(groupBy ...string)` →
`Group(groupBy ...any)` to support Raw SQL expressions
2. **Auto-prefixing logic**: When JOINs are detected (by checking for "
JOIN " in the tables string), unqualified column names are automatically
prefixed with the primary table name
3. **Preserved existing behavior**: Already qualified columns
(containing ".") and Raw expressions are handled as before
4. **Added comprehensive test**: `Test_Model_Group_WithJoin` verifies
the fix works correctly with JOIN queries

---------

Co-authored-by: hailaz <739476267@qq.com>
2025-11-24 15:57:20 +08:00
99d69857fa refactor(database/gdb): add quote for FieldsPrefix (#4485)
Code example:
``` go
	var res *BasicInfo
	err := g.DB().Model("basic_info").
		FieldsPrefix("basic_info", basicInfoColumns).
		Where("id", 35813305356386305).Scan(&res)
	if err != nil {
		panic(err)
	}
	g.Dump(res)
```

SQL generated before modification :
``` sql
SELECT basic_info.id,basic_info.full_name,basic_info.contact FROM `basic_info` WHERE (`id`=35813305356386305) AND `delete_time` IS NULL LIMIT 1
```

SQL generated after modification:
``` sql
SELECT `basic_info`.`id`,`basic_info`.`full_name`,`basic_info`.`contact` FROM `basic_info` WHERE (`id`=35813305356386305) AND `delete_time` IS NULL LIMIT 1
```

---------

Co-authored-by: hailaz <739476267@qq.com>
2025-11-21 17:27:09 +08:00
2d307c5dd1 feat(contrib/drivers/pgsql): add array type numeric[] and decimal[] converting to Go []float64 support #4457 (#4511)
Co-authored-by: hailaz <739476267@qq.com>
2025-11-19 14:35:30 +08:00
1682cc98bb feat(gdb): Allow to set table field metadata and allow to generate table fields registration code when generating dao (#4460)
`gdb`在第一次查询时会拉取一次`table`的`fields meta`信息,为后续orm的字段过滤和时间特性服务,`gen
dao`时已经获得了`table`的所有`fields
meta`,提供一个生成`table`的功能,允许客户自行决定是否生成,是否注入已知表结构缓存到指定`db`实例。
1. 能解决部分兼容`mysql`的二开数据库在获取`fields meta`时无法和`mysql`保持一致的问题。
2.
可以模拟表字段信息而不需要真实的数据库连接,对于已知的表结构,可以直接设置缓存,在无法连接数据库的情况下,仍然可以使用表字段信息,使用gdb构建sql时不需要受限于实际数据库即可使用`gdb.ToSQL()`方法。
4. 提升访问速度

生成的示例目录
<img width="389" height="670" alt="SCR-20251010-ntne"
src="https://github.com/user-attachments/assets/ebb08e70-cce1-4b73-9128-6ff784e4df3b"
/>

生成的示例代码
```golang
// =================================================================================
// This file is auto-generated by the GoFrame CLI tool. You may modify it as needed.
// =================================================================================

package table

import (
	"context"

	"github.com/gogf/gf/v2/database/gdb"
)

// RolePermissions defines the fields of table "role_permissions" with their properties.
// This map is used internally by GoFrame ORM to understand table structure.
var RolePermissions = map[string]*gdb.TableField{
	"role_id": {
		Index:   0,
		Name:    "role_id",
		Type:    "bigint unsigned",
		Null:    false,
		Key:     "PRI",
		Default: nil,
		Extra:   "",
		Comment: "",
	},
	"permission_id": {
		Index:   1,
		Name:    "permission_id",
		Type:    "bigint unsigned",
		Null:    false,
		Key:     "PRI",
		Default: nil,
		Extra:   "",
		Comment: "",
	},
	"created_at": {
		Index:   2,
		Name:    "created_at",
		Type:    "timestamp",
		Null:    false,
		Key:     "",
		Default: "CURRENT_TIMESTAMP",
		Extra:   "DEFAULT_GENERATED",
		Comment: "",
	},
	"updated_at": {
		Index:   3,
		Name:    "updated_at",
		Type:    "timestamp",
		Null:    false,
		Key:     "",
		Default: "CURRENT_TIMESTAMP",
		Extra:   "DEFAULT_GENERATED on update CURRENT_TIMESTAMP",
		Comment: "",
	},
	"deleted_at": {
		Index:   4,
		Name:    "deleted_at",
		Type:    "timestamp",
		Null:    true,
		Key:     "",
		Default: nil,
		Extra:   "",
		Comment: "",
	},
}

// SetRolePermissionsTableFields registers the table fields definition to the database instance.
// db: database instance that implements gdb.DB interface.
// schema: optional schema/namespace name, especially for databases that support schemas.
func SetRolePermissionsTableFields(ctx context.Context, db gdb.DB, schema ...string) error {
	return db.GetCore().SetTableFields(ctx, "role_permissions", RolePermissions, schema...)
}

```
2025-10-15 15:50:16 +08:00
f24729206b fix(database/gdb): Resolved the schema error in the database output log when using the database sharding feature (#4319)
error log
```
2025-06-18T15:36:08.315+08:00 [DEBU] {10a3b0cfd9124a186b89b07f50e67ce6} [  0 ms] [default] [db_0] [rows:1  ] SELECT `id`,`custom_name` FROM `custom` WHERE `id`=1 LIMIT 1

2025-06-18T15:36:09.259+08:00 [DEBU] {10a3b0cfd9124a186b89b07f50e67ce6} [  1 ms] [default] [db_0] [rows:1  ] SELECT `id`,`custom_name`,`remark`FROM `custom` WHERE `id`=2 LIMIT 1

```
right log
```
2025-06-18T15:36:08.315+08:00 [DEBU] {10a3b0cfd9124a186b89b07f50e67ce6} [  0 ms] [default] [db_0] [rows:1  ] SELECT `id`,`custom_name` FROM `custom` WHERE `id`=1 LIMIT 1

2025-06-18T15:36:09.259+08:00 [DEBU] {10a3b0cfd9124a186b89b07f50e67ce6} [  1 ms] [default] [db_1] [rows:1  ] SELECT `id`,`custom_name`,`remark`FROM `custom` WHERE `id`=2 LIMIT 1
```

```

type DbShardingRule struct {
}

func (d *DbShardingRule) SchemaName(ctx context.Context, config gdb.ShardingSchemaConfig, value any) (string, error) {
	if name, ok := value.(string); ok && (len(name) > 0) && gstr.HasPrefix(name, config.Prefix) {
		return name, nil
	}
	return "default", nil
}

func (d *DbShardingRule) TableName(ctx context.Context, config gdb.ShardingTableConfig, value any) (string, error) {
	return "", nil
}
```

```
config := gdb.ShardingConfig{
		Schema: gdb.ShardingSchemaConfig{
			Enable: true,
			Prefix: "db_",
			Rule:   &DbShardingRule{},
		},
	}
dao.Custom.Ctx(ctx).Sharding(config).ShardingValue("db_0").Where("id", 1).One()
dao.Custom.Ctx(ctx).Sharding(config).ShardingValue("db_1").Where("id", 2).One()
```
我有两个完全一样的数据库db_0和db_1,两个custom表里都只有一条数据,id是1和2,执行上面两条查询得的结果是正确的,
输出日志中应该分别是`[default] [db_0]`和`[default] [db_1]`,但是实际输出的都是`[default]
[db_0]`,

查看具体代码实现,该日志由Core实例中获取group和schema构成,而实际执行sql时如果使用了分库特性那么执行sql的link会使用当前面schema重新生成,但是没有重新赋给Core实例,所以输出日志的时候还是错的,只需要在生成link后生成日志前把schema赋给Core就行,方法执行完成后defer将schema重置回去,防止有人重复使用同一个gdb对象造成困扰

![SCR-20250618-ovoc](https://github.com/user-attachments/assets/815c364e-939f-4a2c-9669-d5b7d2742511)

![SCR-20250618-ovvk](https://github.com/user-attachments/assets/e7d0e375-78e6-4748-90ac-d02dba18720f)

![SCR-20250618-ozsu](https://github.com/user-attachments/assets/faa6d69b-331e-476b-8bf8-f62e564b04d3)

![SCR-20250618-ozwj](https://github.com/user-attachments/assets/15c524dc-dc19-4499-a3d3-32bf1d918a3a)
2025-09-28 17:57:27 +08:00
f565a347c4 fix(database/gdb): Fix GetArray return type and add Bools method (#4452)
Change the return type of the GetArray method to Array for consistency
in data structure. Introduce a new Bools method to convert Vars to a
slice of bools, along with corresponding test cases to validate the
functionality.

```go
// 改进前
res, _ := db.Model(table).Fields("id").Array()
ids := make([]int64, 0, len(res))
for _, v := range res {
ids = append(ids, v.Int64())
}
g.Dump(ids)
// 改进后
res, _ := db.Model(table).Fields("id").Array()
ids := res.Int64s()
g.Dump(ids)
```
2025-09-28 17:08:37 +08:00
613
b60b04e27a fix(database/gdb): performance improvement in fields grouping when in… (#4440)
fix https://github.com/gogf/gf/issues/3906

<img width="2604" height="980" alt="image"
src="https://github.com/user-attachments/assets/50852928-7ff5-4676-8ecf-6960c184e805"
/>

---------

Co-authored-by: hailaz <739476267@qq.com>
2025-09-26 11:11:14 +08:00
edc96a8c16 feat(database/gdb): Add the function of obtaining all configurations to facilitate business operations such as verification after addition. (#4389)
…ss operations such as verification after addition.

---------

Signed-off-by: sxp20008 <81209245@qq.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: hailaz <739476267@qq.com>
2025-09-18 16:21:02 +08:00
502d158bc0 fix: version 2.9.2 (#4405) 2025-09-01 15:33:50 +08:00
94cc233325 fix: disable specific staticcheck rules and update lint config (#4396)
fix: disable specific staticcheck rules and update lint config

- Disabled staticcheck rules SA1029, SA1019, S1000, and related checks
in `.golangci.yml` to filter out unwanted linter errors.
- Updated staticcheck checks list for more precise linting control.
- Clarified configuration for easier maintenance and future updates.

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: hailaz <739476267@qq.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-08-29 10:32:30 +08:00
ee24da4e72 refactor: interface{} to any and reflect.Ptr to reflect.Pointer (#4395)
This pull request standardizes the use of the Go 1.18+ `any` type alias
instead of `interface{}` throughout the codebase. The change improves
code readability and aligns with modern Go best practices. The update
touches many files, including core data structures, code generation
templates, logging utilities, and test data, ensuring consistency across
all usages.

**Type alias migration to `any`:**

* Replaced all instances of `interface{}` with `any` in core data
structures such as `garray` and in generated model structs (e.g.,
`TableUser`, `User1`, `User2`) to modernize type usage.
[[1]](diffhunk://#diff-3a1259e160a4dfa5fe49dfe739fbdb986c0d0a2220a709882ea48d3ae1b8f911L31-R31)
[[2]](diffhunk://#diff-6c19859cb32c7516ea95ddc8f8235460818eb2f24d2204308e0d9e1b19e7d90fL15-R19)
[[3]](diffhunk://#diff-a15ba2f5e830b4833c47b902515a4f9e5a4f83a3707698f3229b307ec3776b41L15-R18)
[[4]](diffhunk://#diff-52e0837e84d49221d1b810d88fdf78221f36cffcd664fb42f8aba49a79b974dcL15-R19)
[[5]](diffhunk://#diff-11c3457d1a23a4ca6ecd00d6b856289774936b6a708384cf03aff164044e7546L15-R19)
[[6]](diffhunk://#diff-2cff9cf8e6a0cc34087326d8c8149c3bbaf74c76fdbdf5a73daed13cc04249e1L15-R19)
* Updated function signatures, method parameters, and return types from
`interface{}` to `any` in various parts of the codebase, including code
generation, service logic, and logging utilities (e.g., `mlog`).
[[1]](diffhunk://#diff-175edfeea54490b8fe4e18ffcbea5835efaf8f0b8acf623359073987cae7eb76L48-R55)
[[2]](diffhunk://#diff-2b1953fb78cf3593d8c2c7d911e95b65fd0b847c30ed0b4d167d16fe6d781235L54-R74)
[[3]](diffhunk://#diff-e001b7a4b63603b9b14f00de78a4d570bb76c5f57d856a24643f071032e12356L66-R73)
[[4]](diffhunk://#diff-5582954e8a9983988dc8854ad82067fb2ac6269b988e07357ad8db1dfec5f1a0L39-R41)
[[5]](diffhunk://#diff-c5d51d56f487779a2b6207c7ad26c7a20bbadcc846ce094fe60ab4cabff58c51L107-R107)
[[6]](diffhunk://#diff-f96e6a9fdb416eb1804ceaba1fe0ac637bff22c43837f8bb849c2366ce72d4a1L116-R121)
[[7]](diffhunk://#diff-f94c83a1b08ae060d9346f4a6031fc4a7b9a0b894e02d9afaa09018b6598eac0L112-R112)
[[8]](diffhunk://#diff-748b11dbe8828dd4c040ec23cae0b8fe57ecf0a2d1b7694ea39102294e633c64L36-R36)
[[9]](diffhunk://#diff-748b11dbe8828dd4c040ec23cae0b8fe57ecf0a2d1b7694ea39102294e633c64L74-R74)
[[10]](diffhunk://#diff-748b11dbe8828dd4c040ec23cae0b8fe57ecf0a2d1b7694ea39102294e633c64L96-R96)

**Generated code and templates:**

* Adjusted generated files and code generation templates to output `any`
instead of `interface{}` for relevant struct fields and function
signatures, ensuring that new code generation aligns with the updated
convention.
[[1]](diffhunk://#diff-6c19859cb32c7516ea95ddc8f8235460818eb2f24d2204308e0d9e1b19e7d90fL15-R19)
[[2]](diffhunk://#diff-a15ba2f5e830b4833c47b902515a4f9e5a4f83a3707698f3229b307ec3776b41L15-R18)
[[3]](diffhunk://#diff-52e0837e84d49221d1b810d88fdf78221f36cffcd664fb42f8aba49a79b974dcL15-R19)
[[4]](diffhunk://#diff-11c3457d1a23a4ca6ecd00d6b856289774936b6a708384cf03aff164044e7546L15-R19)
[[5]](diffhunk://#diff-2cff9cf8e6a0cc34087326d8c8149c3bbaf74c76fdbdf5a73daed13cc04249e1L15-R19)
[[6]](diffhunk://#diff-175edfeea54490b8fe4e18ffcbea5835efaf8f0b8acf623359073987cae7eb76L48-R55)
[[7]](diffhunk://#diff-e001b7a4b63603b9b14f00de78a4d570bb76c5f57d856a24643f071032e12356L66-R73)
[[8]](diffhunk://#diff-5582954e8a9983988dc8854ad82067fb2ac6269b988e07357ad8db1dfec5f1a0L39-R41)

**Container and utility updates:**

* Refactored the `garray` container implementation and related
constructors/methods to use `[]any` instead of `[]interface{}`, along
with corresponding function signatures.
[[1]](diffhunk://#diff-3a1259e160a4dfa5fe49dfe739fbdb986c0d0a2220a709882ea48d3ae1b8f911L31-R31)
[[2]](diffhunk://#diff-3a1259e160a4dfa5fe49dfe739fbdb986c0d0a2220a709882ea48d3ae1b8f911L52-R52)
[[3]](diffhunk://#diff-3a1259e160a4dfa5fe49dfe739fbdb986c0d0a2220a709882ea48d3ae1b8f911L62-R62)
[[4]](diffhunk://#diff-3a1259e160a4dfa5fe49dfe739fbdb986c0d0a2220a709882ea48d3ae1b8f911L73-R86)
[[5]](diffhunk://#diff-3a1259e160a4dfa5fe49dfe739fbdb986c0d0a2220a709882ea48d3ae1b8f911L96-R97)
[[6]](diffhunk://#diff-3a1259e160a4dfa5fe49dfe739fbdb986c0d0a2220a709882ea48d3ae1b8f911L107-R114)
[[7]](diffhunk://#diff-3a1259e160a4dfa5fe49dfe739fbdb986c0d0a2220a709882ea48d3ae1b8f911L124-R124)
[[8]](diffhunk://#diff-3a1259e160a4dfa5fe49dfe739fbdb986c0d0a2220a709882ea48d3ae1b8f911L135-R143)
[[9]](diffhunk://#diff-3a1259e160a4dfa5fe49dfe739fbdb986c0d0a2220a709882ea48d3ae1b8f911L167-R167)

These changes collectively modernize the codebase and prepare it for
future Go developments by using the idiomatic `any` type.
2025-08-28 16:53:19 +08:00
8cff64915b fix(tracing): set database span kind to client (#4334)
In [OpenTelemetry
spec](https://opentelemetry.io/docs/specs/semconv/database/database-spans/),
the span kind of database should be `client`.
2025-08-22 22:49:13 +08:00
bec98e8de0 fix(database/gdb): clickhouse can not support int128/int256/uint128/uint256 (#4370)
When use clickhouse and use field type int128/int256/uint128/uint256. It
can not work well and it will return 0 value.
Add the new field types to let it work well.

By the way, the data struct field need be set to big.Int when use the
types.
2025-08-22 22:05:15 +08:00
656ae070da fix(cmd/gf): fix gen sharding dao in multiple shardingPattern tables … (#4379)
fix #4378 
fix #4330

---------

Co-authored-by: hailaz <739476267@qq.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-08-22 16:46:23 +08:00
2d5fcd73cb chore: upgrade dependencies to latest versions and fix security vulne… (#4237)
This PR includes the following updates and fixes:

- **Dependency upgrades**: Updated all dependencies in `go.mod` to their
latest versions to ensure compatibility and leverage the latest features
and fixes.
- **Security fixes**:
- Resolved known vulnerabilities in `golang.org/x/net` by upgrading to
the latest secure version.
- Addressed security issues in `golang.org/x/crypto` by upgrading to the
latest secure version.

These changes improve the overall security and stability of the project.
Please review the changes and ensure compatibility with the updated
dependencies.

---------

Co-authored-by: hailaz <739476267@qq.com>
2025-08-22 15:29:16 +08:00
7ffdff37e4 chore: upgrade golangci-lint configuration and optimize codebase (#4236)
This PR includes the following changes:

- **Upgrade `.golangci.yml`**: Updated the configuration file to align
with the latest golangci-lint version, ensuring compatibility and
leveraging new features.
- **Refactor GitHub Action workflow**: Modified `golangci-lint.yml` in
the GitHub Actions workflow to reflect the updated configuration and
improve CI performance.
- **Codebase optimization**: Refactored code to address issues and
warnings raised by the updated golangci-lint rules, including:
  - Improved function length and complexity.
  - Enhanced error handling and variable naming conventions.
- Fixed minor issues such as unused imports and formatting
inconsistencies.

These changes aim to maintain code quality, ensure compatibility with
the latest tools, and improve overall maintainability.
2025-08-22 13:29:09 +08:00
3d9cdb8997 fix(gdb): support multiple order fields in "with" 2025-06-20 19:11:01 +08:00
9a61a6970f fix(database/gdb): fix transaction propagation feature (#4199) 2025-03-17 14:50:07 +08:00
bc1e1019c5 refract(util/gconv): change Converter interface definition for more convenient usage (#4202) 2025-03-14 18:23:07 +08:00
dfe088f5cd refactor(util/gconv): add Converter feature for more flexable and extensible type converting (#4107) 2025-03-06 23:04:26 +08:00
a3b3c656d9 feat(database/gdb): enable transaction propagation when using tx.GetCtx() after Begin (#4121) 2025-02-11 16:09:27 +08:00
80f57d1c24 fix(database/gdb): gdb.Counter not work in OnDuplicate (#4073) 2024-12-26 18:18:35 +08:00
4c2a78b7bf fix(database/gdb): regular expression pattern for link configuration to be compitable with tidbcloud (#4064) 2024-12-19 21:44:36 +08:00
92eab81926 fix(database/gdb): add compatibility for old configiration with both Type and part of Link configurations (#4058) 2024-12-18 15:54:27 +08:00
f79aef6669 fix(database/gdb): fix context canceled error in transaction due to usage of TransTimeout configuration (#4037) 2024-12-17 21:15:54 +08:00
ced4b57991 fix(contrib/drivers/pgsql): incompatible placeholder replacement with old version (#4036) 2024-12-13 09:29:34 +08:00
b7c74c9a35 fix(contrib/drivers/pgsql): add unix socket connection support (#4028) 2024-12-11 09:58:26 +08:00
13bc192e36 feat(database/gdb): add sharding feature for schema and table (#4014) 2024-12-09 23:12:20 +08:00
5e47590165 feat(database/gdb): add WhereExists/WhereNotExists (#4015) 2024-12-09 09:24:48 +08:00
3cffa4d5d6 fix(database/gdb): CRUD typos (#4017) 2024-12-09 09:22:52 +08:00
b0b84a3937 ci(gci/import): improve golangci.yml and add gci linter (#4010) 2024-12-07 14:17:33 +08:00
2066aa4803 feat(database/gdb): add transaction propagation&isolation level&readonly features (#4013) 2024-12-07 14:01:31 +08:00
ba968949f7 fix(database/gdb): orm tag from embedded struct is missing in with feature (#4011) 2024-12-05 22:02:47 +08:00
4ad061faff feat(contrib/drivers/pgsql): add array type varchar[] and text[] converting to Go []string support (#4000) 2024-12-05 15:50:59 +08:00
532e665841 fix(database/gdb): move Raw parameter from args to sql statement before committed to db driver (#3997) 2024-12-03 15:24:49 +08:00
2c916f8222 fix(database/gdb): issue where the Count/Value/Array query logic was incompatible with the old version when users extended the returned result fields using the Select Hook (#3995) 2024-12-01 23:47:51 +08:00
ec6e091882 chore: fix some function names in comment (#3967) 2024-12-01 11:29:09 +08:00
455830b842 test(database/gdb): add more unit testing cases for Raw feature (#3962) 2024-11-23 18:35:02 +08:00
e9ce1bde87 fix(database/gdb): incompatitable statement like Order("id", "dasc") (#3949) 2024-11-21 15:08:29 +08:00
3090fe7f4e fix(database/gdb): recognize tinyint(1)/int(1) as bool (#3943) 2024-11-20 20:21:33 +08:00
910703ec3a chore(database/gdb): comment update for function Model.Order (#3933) 2024-11-18 18:33:59 +08:00