mirror of
https://gitee.com/johng/gf
synced 2026-06-07 02:12:11 +08:00
This pull request introduces significant improvements to the handling of the `Replace` and `Save` operations for multiple database drivers, especially for MSSQL and PostgreSQL. The changes ensure that these operations now auto-detect primary keys when conflict columns are not explicitly provided, improving usability and aligning behavior across drivers. Additionally, the pull request updates related tests to reflect these enhancements and includes some minor documentation and code cleanup. **Key changes:** ### Enhanced Replace/Save Logic for Database Drivers * **MSSQL Driver:** - `Replace` and `Save` operations now auto-detect primary keys if `OnConflict` is not specified, using the `MERGE` statement for upsert functionality. If no primary key is found in the data, a detailed error is returned. [[1]](diffhunk://#diff-87815aa559a927e2de09bd05148f9841dfc06a1b5f3ecc5e3d5fcb80323a87f8L23-R61) [[2]](diffhunk://#diff-87815aa559a927e2de09bd05148f9841dfc06a1b5f3ecc5e3d5fcb80323a87f8L43-L59) - Updated tests to verify that `Replace` correctly updates or inserts records, and that missing conflict columns are properly handled. [[1]](diffhunk://#diff-bdbde9d7d6ee14c795343767b414740c4396f4dd3e97788b1f9d4e615405a42dL141-R151) [[2]](diffhunk://#diff-26338e93e473300b1313936eb0f6826546473793442f24715fa294b595f7a805L2661-R2707) * **PostgreSQL Driver:** - Similar to MSSQL, `Replace` and `Save` now auto-detect primary keys for conflict resolution if `OnConflict` is not set, and treat `Replace` as a `Save` operation. - Adjusted tests to ensure `Save` and `Replace` work as expected, including verifying data replacement and insertion. [[1]](diffhunk://#diff-c22703c37ebb6836c332f7cd2ada570577ba4564fe39886db02f7c2d0e7a2048L93-R93) [[2]](diffhunk://#diff-c22703c37ebb6836c332f7cd2ada570577ba4564fe39886db02f7c2d0e7a2048R102) [[3]](diffhunk://#diff-c22703c37ebb6836c332f7cd2ada570577ba4564fe39886db02f7c2d0e7a2048L110-R130) * **DM Driver:** - Improved conflict detection: now checks that at least one primary key exists in the provided data when `OnConflict` is not specified, and provides clearer error messages. - Refactored to use the core method for primary key detection and removed redundant code. ### Minor Improvements and Documentation * Added clarifying comments to `DoInsert` methods for ClickHouse, DM, MSSQL, Oracle, and PostgreSQL drivers, specifying that the input list must have at least one validated record. [[1]](diffhunk://#diff-f2e003895041ed3c52b91bb8c270696adc3528d77c39d2f7137af3396267444cR19) [[2]](diffhunk://#diff-f51b30e3f0b0f1284b905385a89992efd0de2fe9ff8c5a4062344dfab17d428eR23) [[3]](diffhunk://#diff-87815aa559a927e2de09bd05148f9841dfc06a1b5f3ecc5e3d5fcb80323a87f8L23-R61) [[4]](diffhunk://#diff-f61dac3fcfd5df4a3936cd8743499c8c0fc45f4f5d0f5398ed84a0cb1603202cR24) [[5]](diffhunk://#diff-c1dfed79aaa3a432057d2bd74d270e4b4094ebcf72984f1161d4972bea009410R16-R72) * Minor code and comment cleanups, including improved formatting and error handling. [[1]](diffhunk://#diff-f61dac3fcfd5df4a3936cd8743499c8c0fc45f4f5d0f5398ed84a0cb1603202cR37) [[2]](diffhunk://#diff-f61dac3fcfd5df4a3936cd8743499c8c0fc45f4f5d0f5398ed84a0cb1603202cL96-R98) [[3]](diffhunk://#diff-f61dac3fcfd5df4a3936cd8743499c8c0fc45f4f5d0f5398ed84a0cb1603202cL106-L116) [[4]](diffhunk://#diff-a17b44c76aaac53d1f164a2bb9440a5531659f4355e7ccfabdadff8dc8633c09L170-R171) [[5]](diffhunk://#diff-56189fa9ae1df51716b50d34d7fe56bfe67a330e8ac2c6b0de7b958db6817ed5R83-R98) ### Workflow and Documentation Updates * Updated example Docker commands in the CI workflow for consistency and clarity. [[1]](diffhunk://#diff-a1a3cb9bdeb5541d148091d973cf266aa3b317e6415a86630e816cbe27cf8b9cL57-R57) [[2]](diffhunk://#diff-a1a3cb9bdeb5541d148091d973cf266aa3b317e6415a86630e816cbe27cf8b9cL78-R78) [[3]](diffhunk://#diff-a1a3cb9bdeb5541d148091d973cf266aa3b317e6415a86630e816cbe27cf8b9cL92-R92) [[4]](diffhunk://#diff-a1a3cb9bdeb5541d148091d973cf266aa3b317e6415a86630e816cbe27cf8b9cL106-R106) [[5]](diffhunk://#diff-a1a3cb9bdeb5541d148091d973cf266aa3b317e6415a86630e816cbe27cf8b9cL153-R153) [[6]](diffhunk://#diff-a1a3cb9bdeb5541d148091d973cf266aa3b317e6415a86630e816cbe27cf8b9cL164-R164) * Removed outdated note about `Replace` support from the SQLite driver documentation. These changes improve the consistency, reliability, and developer experience when performing upsert operations across different database backends. --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Lance Add <1196661499@qq.com>
290 lines
8.6 KiB
YAML
290 lines
8.6 KiB
YAML
# The main codes build and unit testing running workflow.
|
|
name: GoFrame Main CI
|
|
|
|
|
|
on:
|
|
push:
|
|
branches:
|
|
- master
|
|
- develop
|
|
- personal/**
|
|
- feature/**
|
|
- enhance/**
|
|
- fix/**
|
|
|
|
pull_request:
|
|
branches:
|
|
- master
|
|
- develop
|
|
- personal/**
|
|
- feature/**
|
|
- enhance/**
|
|
- fix/**
|
|
workflow_dispatch:
|
|
inputs:
|
|
debug:
|
|
type: boolean
|
|
description: 'Enable tmate Debug'
|
|
required: false
|
|
default: false
|
|
|
|
# This allows a subsequently queued workflow run to interrupt previous runs
|
|
concurrency:
|
|
group: '${{ github.workflow }} @ ${{ github.event.pull_request.head.label || github.head_ref || github.ref }}'
|
|
cancel-in-progress: true
|
|
|
|
env:
|
|
TZ: "Asia/Shanghai"
|
|
# for unit testing cases of some components that only execute on the latest go version.
|
|
LATEST_GO_VERSION: "1.25"
|
|
|
|
jobs:
|
|
code-test:
|
|
strategy:
|
|
matrix:
|
|
# 🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥
|
|
# When adding new go version to the list, make sure:
|
|
# 1. Update the `LATEST_GO_VERSION` env variable.
|
|
# 🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥
|
|
go-version: [ "1.23", "1.24", "1.25" ]
|
|
goarch: [ "386", "amd64" ]
|
|
|
|
runs-on: ubuntu-latest
|
|
|
|
# Service containers to run with `code-test`
|
|
services:
|
|
# Etcd service.
|
|
# docker run -p 2379:2379 -e ALLOW_NONE_AUTHENTICATION=yes bitnamilegacy/etcd:3.4.24
|
|
etcd:
|
|
image: bitnamilegacy/etcd:3.4.24
|
|
env:
|
|
ALLOW_NONE_AUTHENTICATION: yes
|
|
ports:
|
|
- 2379:2379
|
|
|
|
# Redis backend server.
|
|
redis:
|
|
image : redis:7.0
|
|
options: >-
|
|
--health-cmd "redis-cli ping"
|
|
--health-interval 10s
|
|
--health-timeout 5s
|
|
--health-retries 5
|
|
ports:
|
|
# Maps tcp port 6379 on service container to the host
|
|
- 6379:6379
|
|
|
|
# MySQL backend server.
|
|
# docker run \
|
|
# -p 3306:3306 \
|
|
# -e MYSQL_DATABASE=test \
|
|
# -e MYSQL_ROOT_PASSWORD=12345678 \
|
|
# mysql:5.7
|
|
mysql:
|
|
image: mysql:5.7
|
|
env:
|
|
MYSQL_DATABASE : test
|
|
MYSQL_ROOT_PASSWORD: 12345678
|
|
ports:
|
|
- 3306:3306
|
|
|
|
# MariaDb backend server.
|
|
# docker run \
|
|
# -p 3307:3306 \
|
|
# -e MYSQL_DATABASE=test \
|
|
# -e MYSQL_ROOT_PASSWORD=12345678 \
|
|
# mariadb:11.4
|
|
mariadb:
|
|
image: mariadb:11.4
|
|
env:
|
|
MARIADB_DATABASE: test
|
|
MARIADB_ROOT_PASSWORD: 12345678
|
|
ports:
|
|
- 3307:3306
|
|
|
|
# PostgreSQL backend server.
|
|
# docker run \
|
|
# -p 5432:5432 \
|
|
# -e POSTGRES_PASSWORD=12345678 \
|
|
# -e POSTGRES_USER=postgres \
|
|
# -e POSTGRES_DB=test \
|
|
# -v postgres:/Users/john/Temp/postgresql/data \
|
|
# postgres:17-alpine
|
|
postgres:
|
|
image: postgres:17-alpine
|
|
env:
|
|
POSTGRES_PASSWORD: 12345678
|
|
POSTGRES_USER: postgres
|
|
POSTGRES_DB: test
|
|
TZ: Asia/Shanghai
|
|
ports:
|
|
- 5432:5432
|
|
# Set health checks to wait until postgres has started
|
|
options: >-
|
|
--health-cmd pg_isready
|
|
--health-interval 10s
|
|
--health-timeout 5s
|
|
--health-retries 5
|
|
|
|
# MSSQL backend server.
|
|
# docker run \
|
|
# -p 1433:1433 \
|
|
# -e ACCEPT_EULA=Y \
|
|
# -e SA_PASSWORD=LoremIpsum86 \
|
|
# -e MSSQL_USER=root \
|
|
# -e MSSQL_PASSWORD=LoremIpsum86 \
|
|
# mcr.microsoft.com/mssql/server:2022-latest
|
|
mssql:
|
|
image: mcr.microsoft.com/mssql/server:2022-latest
|
|
env:
|
|
TZ: Asia/Shanghai
|
|
ACCEPT_EULA: Y
|
|
MSSQL_SA_PASSWORD: LoremIpsum86
|
|
ports:
|
|
- 1433:1433
|
|
options: >-
|
|
--health-cmd="/opt/mssql-tools18/bin/sqlcmd -S localhost -U sa -P ${MSSQL_SA_PASSWORD} -N -C -l 30 -Q \"SELECT 1\" || exit 1"
|
|
--health-start-period 10s
|
|
--health-interval 10s
|
|
--health-timeout 5s
|
|
--health-retries 10
|
|
|
|
# ClickHouse backend server.
|
|
# docker run \
|
|
# -p 9000:9000 -p 8123:8123 -p 9001:9001 \
|
|
# clickhouse/clickhouse-server:24.11.1.2557-alpine
|
|
clickhouse-server:
|
|
image: clickhouse/clickhouse-server:24.11.1.2557-alpine
|
|
ports:
|
|
- 9000:9000
|
|
- 8123:8123
|
|
- 9001:9001
|
|
|
|
# Polaris backend server.
|
|
# docker run \
|
|
# -p 8090:8090 -p 8091:8091 -p 8093:8093 -p 9090:9090 -p 9091:9091 \
|
|
# polarismesh/polaris-standalone:v1.17.2
|
|
polaris:
|
|
image: polarismesh/polaris-standalone:v1.17.2
|
|
ports:
|
|
- 8090:8090
|
|
- 8091:8091
|
|
- 8093:8093
|
|
- 9090:9090
|
|
- 9091:9091
|
|
|
|
# Oracle 11g server.
|
|
# docker run \
|
|
# -e ORACLE_ALLOW_REMOTE=true \
|
|
# -e ORACLE_SID=XE \
|
|
# -e ORACLE_DB_USER_NAME=system \
|
|
# -e ORACLE_DB_PASSWORD=oracle \
|
|
# -p 1521:1521 \
|
|
# loads/oracle-xe-11g-r2:11.2.0
|
|
oracle-server:
|
|
image: loads/oracle-xe-11g-r2:11.2.0
|
|
env:
|
|
ORACLE_ALLOW_REMOTE: true
|
|
ORACLE_SID: XE
|
|
ORACLE_DB_USER_NAME: system
|
|
ORACLE_DB_PASSWORD: oracle
|
|
ports:
|
|
- 1521:1521
|
|
|
|
# dm8 server
|
|
# docker run -p 5236:5236 loads/dm:v8.1.2.128_ent_x86_64_ctm_pack4
|
|
dm-server:
|
|
image: loads/dm:v8.1.2.128_ent_x86_64_ctm_pack4
|
|
ports:
|
|
- 5236:5236
|
|
|
|
zookeeper:
|
|
image: zookeeper:3.8
|
|
ports:
|
|
- 2181:2181
|
|
|
|
steps:
|
|
# TODO: szenius/set-timezone update to node16
|
|
# sudo ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
|
|
- name: Setup Timezone
|
|
uses: szenius/set-timezone@v2.0
|
|
with:
|
|
timezoneLinux: "Asia/Shanghai"
|
|
|
|
- name: Checkout Repository
|
|
uses: actions/checkout@v5
|
|
|
|
- name: Setup tmate Session
|
|
uses: mxschmitt/action-tmate@v3
|
|
if: ${{ github.event_name == 'workflow_dispatch' && inputs.debug }}
|
|
with:
|
|
detached: true
|
|
limit-access-to-actor: false
|
|
|
|
- name: Free Disk Space
|
|
run: |
|
|
df -h /
|
|
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/hostedtoolcache/CodeQL /opt/hostedtoolcache/Python || true
|
|
df -h /
|
|
|
|
- name: Start Apollo Containers
|
|
run: docker compose -f ".github/workflows/apollo/docker-compose.yml" up -d --build
|
|
|
|
- name: Start Nacos Containers
|
|
run: docker compose -f ".github/workflows/nacos/docker-compose.yml" up -d --build
|
|
|
|
- name: Start Redis Cluster Containers
|
|
run: docker compose -f ".github/workflows/redis/docker-compose.yml" up -d --build
|
|
|
|
- name: Start Consul Containers
|
|
run: docker compose -f ".github/workflows/consul/docker-compose.yml" up -d --build
|
|
|
|
- name: Setup Golang ${{ matrix.go-version }}
|
|
uses: actions/setup-go@v5
|
|
with:
|
|
go-version: ${{ matrix.go-version }}
|
|
cache-dependency-path: '**/go.sum'
|
|
|
|
- name: Install Protoc
|
|
uses: arduino/setup-protoc@v3
|
|
with:
|
|
version: "31.x"
|
|
repo-token: ${{ secrets.GITHUB_TOKEN }}
|
|
|
|
- name: Install the protocol compiler plugins for Go
|
|
run: |
|
|
go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
|
|
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest
|
|
export PATH="$PATH:$(go env GOPATH)/bin"
|
|
|
|
- name: Before Script
|
|
run: bash .github/workflows/scripts/before_script.sh
|
|
|
|
- name: Build & Test
|
|
if: ${{ (github.event_name == 'push' && github.ref != 'refs/heads/master') || github.event_name == 'pull_request' }}
|
|
run: bash .github/workflows/scripts/ci-main.sh
|
|
|
|
- name: Build & Test & Coverage
|
|
if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/master' }}
|
|
run: bash .github/workflows/scripts/ci-main.sh coverage
|
|
|
|
- name: Stop Redis Cluster Containers
|
|
run: docker compose -f ".github/workflows/redis/docker-compose.yml" down
|
|
|
|
- name: Stop Apollo Containers
|
|
run: docker compose -f ".github/workflows/apollo/docker-compose.yml" down
|
|
|
|
- name: Stop Nacos Containers
|
|
run: docker compose -f ".github/workflows/nacos/docker-compose.yml" down
|
|
|
|
- name: Stop Consul Containers
|
|
run: docker compose -f ".github/workflows/consul/docker-compose.yml" down
|
|
|
|
- name: Report Coverage
|
|
uses: codecov/codecov-action@v4
|
|
# Only report coverage on the latest go version
|
|
if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/master' && matrix.go-version == env.LATEST_GO_VERSION }}
|
|
with:
|
|
flags: go-${{ matrix.go-version }}-${{ matrix.goarch }}
|
|
token: ${{ secrets.CODECOV_TOKEN }}
|