Compare commits

..

53 Commits

Author SHA1 Message Date
06fc786416 fix issue of gcache 2018-12-14 18:38:29 +08:00
c003a92408 remove HANGUP signal handle of Web Server admin 2018-12-14 10:11:05 +08:00
c961c22cd7 update gview: show empty string when the variable does not exist 2018-12-13 18:24:22 +08:00
105a821069 fix issue: database connection pool does not work expectly 2018-12-12 20:01:10 +08:00
670993f769 remove auto-adding main package path to search path of WebServer in develop environment 2018-12-11 19:30:38 +08:00
60a571f291 Fix issue: add mutex lock for method Remove of garray. 2018-12-11 09:56:58 +08:00
20a0cb2cd9 Merge branch 'master' of https://gitee.com/johng/gf 2018-12-10 20:24:24 +08:00
51e70be04d Add rewrite feature to WebServer; Add Unique for un sorted array of garray; Fix map conversion issue of gjson. 2018-12-10 20:24:20 +08:00
ac65b808c6 !14 修复mssql的OPEN方法BUG
Merge pull request !14 from 蚊子/master
2018-12-10 18:43:56 +08:00
eb9ddf3c47 修改mssql的open方法连接串 2018-12-10 18:14:51 +08:00
80993e9f77 Fix issue of gjson package in map conversion 2018-12-09 22:30:10 +08:00
b7a6d257d5 Fix binary decoding issue of gbinary package. 2018-12-08 18:18:05 +08:00
7022486e93 Package gview: build-in function 'date' supports printing current datetime. 2018-12-05 15:52:38 +08:00
83f5a9d34e Package gview: build-in function 'date' supports printing current datetime. 2018-12-05 15:48:15 +08:00
431e1051b8 Fix gdb.Model.Count method: Count with no fields checking 2018-12-05 13:52:34 +08:00
f8ab5c3842 VERSION updates 2018-12-04 20:29:32 +08:00
101d095f45 GF greets you. 2018-12-04 20:28:06 +08:00
8481de2b47 Add method for gcache package, to show all map datas of the cache. 2018-12-04 20:23:48 +08:00
c973f133de hot fix 2018-12-04 19:50:24 +08:00
d23cdcbe57 改进gcache,完善功能及基准测试;改进gconv.Map对struct转换时默认使用json tag作为键名 2018-12-04 19:26:46 +08:00
9a52175bd6 gredis增加GetConn方法获取原生redis连接对象 2018-12-01 11:28:47 +08:00
4275218841 README updates 2018-11-30 20:43:08 +08:00
663a2c2a16 README updates 2018-11-30 20:40:23 +08:00
da58a60ad5 README updates 2018-11-30 20:38:53 +08:00
2c26063f4b README updates 2018-11-30 20:37:28 +08:00
b19e47783b ghttp增加静态文件目录映射功能;改进gspath目录管理功能;改进gconv的slice转换功能,并增加gconv.Map方法 2018-11-30 09:48:57 +08:00
aee266eea0 WebServer改进 2018-11-28 20:19:28 +08:00
8304769953 代码修正 2018-11-27 20:37:57 +08:00
914a74bca9 完善代码示例 2018-11-26 09:33:45 +08:00
b965dbff70 gvar调用端改进,去掉不必要的并发安全参数;错误提示细节改进 2018-11-25 22:18:36 +08:00
9f9bcd2467 RELEASE updates 2018-11-24 17:21:30 +08:00
b3353afe3c WebServer添加RouterCacheExpire配置参数 2018-11-24 11:55:57 +08:00
4e3081afee ORM增加mysql datetime参数写入示例 2018-11-24 09:42:21 +08:00
578a6a2df3 改进随机数生成缓冲区 2018-11-24 08:43:39 +08:00
aa42ddd3f1 改进随机数生成缓冲区 2018-11-23 21:39:05 +08:00
69738c337f VERSION updates 2018-11-23 16:50:58 +08:00
12f099fd54 VERSION updates, TODO++ 2018-11-23 16:47:03 +08:00
38932f306d 改进gspath缓存数据结构 2018-11-23 16:45:30 +08:00
5e7e1077a1 性能改进 2018-11-23 09:20:45 +08:00
8f85311332 完善获取数据库配置失败时的错误提示 2018-11-21 00:02:29 +08:00
6eb2887a5a README updates 2018-11-20 23:26:58 +08:00
54f4fd3101 README updates 2018-11-20 10:15:23 +08:00
64a22acf84 WebServer允许同一HOOK事件被多次绑定注册,先注册的回调函数优先级更高 2018-11-19 23:13:12 +08:00
4e5877923d dev 2018-11-19 21:49:43 +08:00
ceaa1a4dd1 Merge branch 'develop' of https://gitee.com/johng/gf into develop 2018-11-19 21:12:21 +08:00
9e1ad46c90 改进ghttp.Request,增加SetParam/GetParam请求流程自定义变量方法;gvar模块增加VarRead只读接口 2018-11-19 21:11:43 +08:00
d5a3fefd8b !13 ORM新增对MSSQL的支持
Merge pull request !13 from 蚊子/master
2018-11-19 11:48:18 +08:00
d85332aca1 ORM新增对MSSQL的支持 2018-11-19 11:38:57 +08:00
10c3f6d85a 完善程序细节和测试 2018-11-18 22:22:44 +08:00
ea4764f1f9 当前工作目录为系统临时目录时,gcfg/gview/ghttp模块默认不添加工作目录到搜索路径 2018-11-18 19:45:04 +08:00
fe753b0bc8 当前工作目录为系统临时目录时,gcfg/gview/ghttp模块默认不添加工作目录到搜索路径 2018-11-18 19:37:42 +08:00
04608269fe 修复gspath模块是windows下搜索失效问题 2018-11-18 19:14:17 +08:00
6addd64cf0 glog模块日志前缀输出改进 2018-11-17 22:12:41 +08:00
130 changed files with 8170 additions and 1400 deletions

1
.gitignore vendored
View File

@ -16,3 +16,4 @@ cbuild
**/.DS_Store
.vscode/
go.sum

View File

@ -18,4 +18,4 @@ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
SOFTWARE.

169
README.MD
View File

@ -1,49 +1,29 @@
<div align=center>
<img src="http://cover.kancloud.cn/johng/gf" width="150"/>
<img src="https://gfer.me/cover.png" width="150"/>
</div>
`GF(GoFrame)` is a modular, lightweight, loosely coupled, high performance application development framework written in Go. Supporting graceful server, hot updates, multi-domain, multi-port, multi-service, HTTP/HTTPS, dynamic/hook routing and many more features. Providing a series of core components and dozens of practical modules.
GF(Go Frame)是一款模块化、松耦合、轻量级、高性能的Go语言Web开发框架。支持热重启、热更新、多域名、多端口、多服务、HTTP/HTTPS、动态路由等特性
并提供了Web服务开发的系列核心组件Router、Cookie、Session、服务注册、配置管理、模板引擎、数据校验、分页管理、数据库ORM等等等等
并且提供了数十个实用开发模块集缓存、日志、时间、命令行、二进制、文件锁、对象池、连接池、数据编码、进程管理、进程通信、TCP/UDP组件、
并发安全容器、Goroutine池等等等等等等。
开源项目地址(仓库保持实时同步)
[Gitee](https://gitee.com/johng/gf)[Github](https://github.com/johng-cn/gf)。
使用中有任何问题/建议欢迎加入技术QQ群交流**116707870**。
如有优秀的框架使用案例,欢迎联系作者将地址展示到项目库中,您的牛逼将被世人所瞻仰。
# 安装
```html
# Installation
```
go get -u gitee.com/johng/gf
```
# 限制
```shell
golang版本 >= 1.9.2
# Limitation
```
golang version >= 1.9.2
```
# 特点
1. 轻量级、高性能,模块化、松耦合设计,丰富的开发模块;
1. 热重启、热更新特性并支持Web界面及命令行管理接口
1. 专业的技术交流群,完善的开发文档及示例代码,良好的中文化支持;
1. 支持多种形式的服务注册特性,灵活高效的路由控制管理;
1. 支持服务事件回调注册功能可供选择的pprof性能分析模块
1. 支持配置文件及模板文件的自动检测更新机制,即修改即生效;
1. 支持自定义日期时间格式的时间模块类似PHP日期时间格式化
1. 强大的数据/表单校验模块支持常用的40种及自定义校验规则
1. 强大的网络通信TCP/UDP组件并提供TCP连接池特性简便高效
1. 提供了对基本数据类型的并发安全封装,提供了常用的数据结构容器;
1. 支持Go变量/Json/Xml/Yml/Toml任意数据格式之间的相互转换及创建
1. 强大的数据库ORM支持应用层级的集群管理、读写分离、负载均衡查询缓存、方法及链式ORM操作
1. 更多特点请查阅框架[手册](https://gfer.me)和[源码](https://godoc.org/github.com/johng-cn/gf)
# Documentation
# 文档
GoFrame开发文档[gfer.me](https://gfer.me)
* [中文文档](https://gfer.me/)
# Architecture
<div align=center>
<img src="https://gfer.me/images/arch.png"/>
</div>
# Quick Start
# 使用
## Hello World
```go
package main
@ -60,122 +40,7 @@ func main() {
s.Run()
}
```
## 多域名支持
```go
package main
import (
"gitee.com/johng/gf/g"
"gitee.com/johng/gf/g/net/ghttp"
)
# License
func main() {
s := g.Server()
s.Domain("localhost1,localhost2").BindHandler("/", func(r *ghttp.Request) {
r.Response.Write("localhostx")
})
s.Run()
}
```
## 多端口支持
```go
package main
import (
"gitee.com/johng/gf/g"
"gitee.com/johng/gf/g/net/ghttp"
)
func main() {
s := g.Server()
s.BindHandler("/", func(r *ghttp.Request){
r.Response.Writeln("go frame!")
})
s.SetPort(8080, 8081, 8082)
s.Run()
}
```
## 路由控制
```go
package main
import (
"gitee.com/johng/gf/g"
"gitee.com/johng/gf/g/net/ghttp"
)
func main() {
s := g.Server()
s.BindHandler("/order/:action/{page}.html", func(r *ghttp.Request){
r.Response.Writef("action:%s, page:%s", r.Get("action"), r.Get("page"))
})
s.SetPort(8199)
s.Run()
}
```
## 数据库ORM
### ORM创建/关闭
```go
// 获取默认配置的单例数据库对象(配置名称为"default")
db, err := gdb.DB()
// 获取配置分组名称为"user-center"的单例数据库对象
db, err := gdb.DB("user-center")
// 无须显示Close数据库引擎底层采用了链接池设计当链接不再使用时会自动关闭
```
### 单表/联表查询
```go
// 查询多条记录并使用Limit分页
r, err := db.Table("user").Where("u.uid > ?", 1).Limit(0, 10).Select()
// 查询符合条件的单条记录(第一条)
r, err := db.Table("user u").LeftJoin("user_detail ud", "u.uid=ud.uid").Fields("u.*,ud.site").Where("u.uid=?", 1).One()
// 查询指定字段值
r, err := db.Table("user u").RightJoin("user_detail ud", "u.uid=ud.uid").Fields("ud.site").Where("u.uid=?", 1).Value()
// 分组及排序
r, err := db.Table("user u").InnerJoin("user_detail ud", "u.uid=ud.uid").Fields("u.*,ud.city").GroupBy("city").OrderBy("register_time asc").Select()
// 不使用john的联表查询
r, err := db.Table("user u,user_detail ud").Where("u.uid=ud.uid").Fields("u.*,ud.city").All()
// 不使用Fields方法指定查询字段时默认查询为"*"
r, err := db.Table("user").Where("u.uid=1",).One()
```
### 更新/删除
```go
// 更新
r, err := db.Table("user").Data(gdb.Map{"name" : "john2"}).Where("name=?", "john").Update()
r, err := db.Table("user").Data("name='john3'").Where("name=?", "john2").Update()
// 删除
r, err := db.Table("user").Where("uid=?", 10).Delete()
// Data数值方法的参数形式比较灵活
r, err := db.Table("user").Data(`name="john"`).Update()
r, err := db.Table("user").Data("name", "john").Update()
r, err := db.Table("user").Data(g.Map{"name" : "john"}).Update()
```
### 写入/保存
```go
r, err := db.Table("user").Data(gdb.Map{"name": "john"}).Insert()
r, err := db.Table("user").Data(gdb.Map{"uid": 10000, "name": "john"}).Replace()
r, err := db.Table("user").Data(gdb.Map{"uid": 10001, "name": "john"}).Save()
```
### 事务操作
```go
if tx, err := db.Begin(); err == nil {
if r, err := tx.Table("user").Data(gdb.Map{"uid":1, "name": "john"}).Save(); err == nil {
tx.Commit()
} else {
tx.Rollback()
}
fmt.Println(r, err)
}
```
...
更多特性及示例请查看官方开发文档:[gfer.me](https://gfer.me)
GF is licensed under the [MIT License](LICENSE), 100% free and open-source.

47
README_ZH.MD Normal file
View File

@ -0,0 +1,47 @@
<div align=center>
<img src="https://gfer.me/cover.png" width="150"/>
</div>
`GF(Go Frame)`是一款模块化、松耦合、轻量级、高性能的Go应用开发框架。支持热重启、热更新、多域名、多端口、多服务、HTTP/HTTPS、动态路由等特性
并提供了Web服务开发的系列核心组件Router、Cookie、Session、服务注册、配置管理、模板引擎、数据校验、分页管理、数据库ORM等等等等
并且提供了数十个内置核心开发模块集缓存、日志、时间、命令行、二进制、文件锁、内存锁、对象池、连接池、数据编码、进程管理、进程通信、文件监控、定时任务、TCP/UDP组件、
并发安全容器等等等等等等。
# 安装
```html
go get -u gitee.com/johng/gf
```
# 限制
```shell
golang版本 >= 1.9.2
```
# 架构
<div align=center>
<img src="https://gfer.me/images/arch.png"/>
</div>
# 文档
[https://gfer.me](https://gfer.me)
# 使用
```go
package main
import (
"gitee.com/johng/gf/g"
"gitee.com/johng/gf/g/net/ghttp"
)
func main() {
s := g.Server()
s.BindHandler("/", func(r *ghttp.Request) {
r.Response.Write("Hello World")
})
s.Run()
}
```

View File

@ -1,136 +1,49 @@
# `v0.97.399 beta` (2018-04-23)
1、 增加gfsnotify文件监控模块
2、 配置管理模块增加配置文件自动检测更新机制;
3、 模板引擎增加对模板文件的自动检测更新机制;
4、 改进gconv包基本类型转换功能提高转换性能
5、 增加gpage分页管理包支持动态分页、静态分页以及自定义分页样式特性
6、 ghttp.Request增加Exit方法用以标记服务退出当在服务执行前调用后服务将不再执行
7、 ghttp.Response去掉WriteString方法统一使用Write方法返回数据流是使用灵活的参数形式
8、 模板引擎增加模板变量暴露接口LockFunc/RLockFunc以便支持开发者灵活处理模板变量
9、 ghttp.Server增加access & error log功能并支持开发者自定义日志处理回调函数注册
10、增加gredis包支持对redis的客户端操作封装并将gredis.Redis对象加入到gins单例管理器中进行统一配置管理维护
11、gins单例管理器增加对单例对象配置文件的自动检测更新机制当配置文件在外部发生变更时自动刷新单例管理器中的单例对象
12、gdb数据库ORM包增加And/Or条件链式方法并改进Where/Data方法参数灵活性
13、对于新增加的模块同时也增加了对应的开发文档并梳理完善了现有的其他模块开发文档
14、修复ISSUE:
#IISWI gitee.com/johng/gf/issues/IISWI,
#IISMY gitee.com/johng/gf/issues/IISMY,
反馈并跟踪完成第三方依赖mxj包的ISSUE修复(github.com/clbanning/mxj/issues/48)
# `v0.98.503 beta` (2018-05-21)
# `v1.2.11` (2018-11-26)
## 新特性
1、平滑重启特性( http://gf.johng.cn/625833 )
2、gflock文件锁模块( http://gf.johng.cn/626062 )
3、gproc进程管理及通信模块( http://gf.johng.cn/626063 )
4、gpage分页管理模块强大的动态分页及静态分页功能并为开发者自定义分页样式提供了极高的灵活度( http://gf.johng.cn/597431 )
5、ghttp.Server增加多端口监听特性并支持HTTP/HTTPS( http://gf.johng.cn/494366 , http://gf.johng.cn/598802 )
6、增加gspath目录检索包管理工具支持对多目录下的文件检索特性
7、ghttp包控制器及执行对象注册增加更灵活的动态路由特性路由规则增加{method}变量支持
1. `ORM`新增对`SQLServer`及`Oracle`的支持([https://gfer.me/database/orm/database](https://gfer.me/database/orm/database))
1. 完成`gvalid`模块校验结果的顺序特性([https://gfer.me/util/gvalid/checkmap](https://gfer.me/util/gvalid/checkmap));
1. 改进`ghttp.Request.Exit`,使得调用该方法时立即退出业务执行,开发者无需调用`Exit`方法时再使用`return`返回([https://gfer.me/net/ghttp/service/object](https://gfer.me/net/ghttp/service/object))
1. 模板引擎新增若干内置函数:`text/html/htmldecode/url/urldecode/date/compare/substr/strlimit/hidestr/highlight/toupper/tolower/nl2br` ([https://gfer.me/os/gview/funcs](https://gfer.me/os/gview/funcs));
1. 模板引擎新增内置变量`Config` ([https://gfer.me/os/gview/vars](https://gfer.me/os/gview/vars));
1. 改进`gconv.Struct`转换默认规则,支持不区分大小写的键名与属性名称匹配
1. `gform`配置文件支持`linkinfo`自定义数据库连接字段([https://gfer.me/database/orm/config](https://gfer.me/database/orm/config))
1. `gfsnotify`模块增加对特定回调的取消注册功能([https://gfer.me/os/gfsnotify/index](https://gfer.me/os/gfsnotify/index)
## 新功能
1、gutil包增加MapToStruct方法支持将map数据类型映射为struct对象
2、gconv
1)、gconv包增加按照类型名称字符串进行类型转换
2)、gconv包新增Time/TimeDuration类型转换方法
3、ghttp
1)、增加Web Server目录安全访问控制机制
2)、ghttp.Server增加自定义状态码回调函数注册处理
4、gdb
1)、gdb包增加gdb.GetStruct/gdb.Model.Struct方法获取查询结果记录自动转换为指定对象
2)、gdb增加Value/Record/Result类型增加对Value类型的系列类型转换方法
3)、gdb包增加db.GetCount,tx.GetCount,model.Count数量查询方法
1. 改进`ghttp.Request`,增加`SetParam/GetParam`请求流程自定义变量设置/获取方法,用于在请求流程中的回调函数共享变量([https://gfer.me/net/ghttp/request](https://gfer.me/net/ghttp/request);
1. 改进`ghttp.Response`,增加`ServeFileDownload`方法用于WebServer引导客户端下载文件([https://gfer.me/net/ghttp/response](https://gfer.me/net/ghttp/response));
1. `gvar`模块新增`gvar.VarRead`只读接口,用于控制对外只暴露数据读取功能
1. 增加`g.Throw`抛异常方法,`g.TryCatch`异常捕获方法封装;
1. 改进`gcron`模块增加自定义的Cron管理对象增加`New/Start/Stop`方法;
## 功能改进
1、改进gredis客户端功能封装
2、改进grand包随机数生成性能
3、grand/gdb/gredis包增加benchmark性能测试脚本
4、改进gjson/gparser包的ToStruct方法实现
5、gdb 改进gdb.New获取ORM操作对象性能
6、gcfg :改进配置文件检索功能;
7、gview模板引擎增加多目录检索功能
8、gfile增加源码main包目录获取方法MainPkgPath
9、ghttp
1)、ghttp.Request增加请求进入和完成时间记录并增加到默认日志内容中
2)、ghttp.Server事件回调之间支持通过ghttp.Request.Param自定义参数进行流程传参
10、gdb
1)、改进gdb.Result与gdb.List, gdb.Record与gdb.Map之间的类型转换便于业务层数据编码处理(如json/xml)
2)、改进gdb.Tx.GetValue返回值类型
3)、gdb.Model.Data参数支持更加灵活的map参数
1. WebServer添加`RouterCacheExpire`配置参数,用于设置路由检索缓存过期时间
1. WebServer允许同一`HOOK`事件被多次绑定注册,先注册的回调函数优先级更高([https://gfer.me/net/ghttp/service/hook](https://gfer.me/net/ghttp/service/hook));
1. 当前工作目录为系统临时目录时,`gcfg`/`gview`/`ghttp`模块默认不添加工作目录到搜索路径;
1. 改进`WebSocket`默认支持跨域请求([https://gfer.me/net/ghttp/websocket](https://gfer.me/net/ghttp/websocket));
1. 改进`gtime.Format`支持中文;
1. 改进`gfsnotify`,支持编辑器对文件非执行标准编辑时(RENAME+CHMOD)的热更新问题;
1. 改进`gtype.Set`方法增加Set原子操作返回旧的变量值;
1. `gfile.ScanDir`增加支持`pattern`多个文件模式匹配,使用'`,`'符号分隔多个匹配模式;
1. `gcfg`模块增加获取配置变量为`*gvar.Var`;
1. `gstr`模块增加对中文截取方法;
1. 改进`gtime.StrToTime`对常用时间格式匹配模式,新增`gtime.ParseTimeFromContent`方法;
1. 修改配置管理、模板引擎、调试模式的环境变量名称为大写下划线标准格式;
1. 改进`grand`模块随机数生成设计,底层使用`crypto/rand`+缓冲区实现高速的随机数生成([https://gfer.me/util/grand/index](https://gfer.me/util/grand/index));
## 问题修复
1、ghttp
1)、修复ghttp包路由缓存问题
2)、修复服务注册时的控制器及执行对象方法丢失问题;
2、gconv
1)、修正gconv.Float64方法位大小设置问题
2)、修复gconv.Int64(float64(xxx))问题;
2、gdb
1)、修复gdb.GetAll针对返回数据列表的for..range...的返回结果slice相同指针问题
2)、修复gdb.Delete方法错误
3)、修复gdb.Model.And/Or方法
4)、修复gdb.Model.Where方法参数处理问题
3、garray修复garray包Remove方法锁机制问题
4、gtype 修复gtype.Float32/gtype.Float64对象类型的方法逻辑错误
5、gfsnotify修复在windows下文件参数中不同文件分隔符引起的热更新机制失效问题
6、修复gvalid包验证问题如果值为nil并且不需要require*验证时,其他验证失效。并增加单元测试项,测试通过。
# `v0.99.682 beta` (2018-08-07)
## 新特性
1、新增gdes包用于DES加密/加密算法处理;
2、新增gkafka包kafka的golang客户端
3、新增gpool对象复用池比较于标准库的sync.Pool更加灵活强大可自定义对象的缓存时间、创建方法、销毁方法(http://gf.johng.cn/686654)
4、完成网络通信gtcp/gudp包的重构并进行了大量的改进工作新增了详尽的开发文档及示例代码(http://gf.johng.cn/494382)
5、增加gring并发安全环标准库container/ring包的并发安全版本并做了易用性的封装(http://gf.johng.cn/686655)
6、gtime包新增了自定义日期格式话的支持格式化语法类似PHP的date语法(http://gf.johng.cn/494387)
7、gdb增加调试模式特性使用SetDebug方法实现在调试模式下可以获取详细的SQL执行记录增加了详细的开发文档及示例代码(http://gf.johng.cn/702801)
8、gdb增加查询缓存特性使用Cache方法实现增加了详细的开发文档及示例代码(http://gf.johng.cn/702801)
9、ghttp.Server路由功能增加字段匹配规则特性支持如/order/list/{page}.html 动态路由规则特性(http://gf.johng.cn/702766)
10、gpage分页包增加分页URL规则生成模板特性内部可使用{.page}变量指定页码位置(http://gf.johng.cn/716438)
11、增加gmap.Map对象这是gmap.InterfaceInterfaceMap的别名
## 新功能
1、gdb增加MaxIdleConnCount/MaxOpenConnCount/MaxConnLifetime三项配置并增加SetMaxConnLifetime方法
2、ghttp.Client增加HTTP账号密码设置功能(SetBasicAuth)
3、glog新增对系统换行符号的自适配调整(\n|\r\n)
4、增加glog控制台调试模式打印开关(SetDebug)
5、gcfg增加SetFileName方法设置默认读取的配置文件名称
6、gcfg/gjson/gparser包新增Int8/16/32/64,Uint8/16/32/64方法
7、增加gzip方法的封装(Zip/Unzip)
8、gview增加模板变量分隔符设置方法SetDelimiters
9、ghttp.Response增加Writef、Writefln方法
## 功能改进
1、改进gfilepool文件指针池设计改进gfile文本内容写入增加指针池使用
2、gdb包增加调试模式特性并支持在调试模式下获得已执行的SQL列表结果
3、改进gproc进程间通信机制增加进程消息分组特性并限定队列大小
4、gdb结果方法处理增加ToXml/ToJson方法
5、gregx包名修改为gregex
6、改进gtime.StrToTime方法新增对常见标准时间日期的自动转换以及对时区的自动识别支持并调整gconv,gvalid对该包的引用
7、增加对字符集转换的封装gxml包中使用新增的字符集转换包来做处理
8、ghttp.Server.EnableAdmin页面Restart接口支持GET参数newExeFilePath支持
9、ghttp.Server平滑重启机制增加可自定义重启可执行文件路径特别是针对windows系统特别有用(因为windows下不支持可执行文件覆盖更新)
10、改进ghttp.Server静态文件检索设计增加开发环境时的main包源码目录查找机制改进gcfg/gview的main包源码目录查找机制
11、优化gcache设计LRU特性非默认开启优化gtype/gcache基准测试脚本新增gregx基准测试脚本改进设计提升性能
12、gfile包增加GoRootOfBuild方法用于获取编译时的GOROOT数值并改进glog包中backtrace的GOROOT路径过滤处理
13、改进grpool代码质量并改进对池化goroutine数量的限制设计
14、改进gdb.Map/List及g.Map/List的类型定义改用别名特性以便支持原生类型输入(map/slice)并修复gdb.Model.Update方法参数处理问题
15、调整ghttp包示例代码目录结构增加ghttp.Client自定义Header方法ghttp.Cookie增加Map方法用于获得客户端提交的所有cookie值构造成map返回
16、删除gcharset中的getcharset方法
17、去掉gmap中常用的基本数据类型转换获取方法
18、改进gconv.String方法当无法使用基本类型进行字符串转换时使用json.Marshal进行转换
19、gvalid.CheckObject方法名称修改为gvalid.CheckStruct
1. 修复`gspath`模块在`windows`下搜索失效问题;
1. 修复`gspath`模块Search时带有indexFiles的检索问题;
1. bug fix INZS1([https://gitee.com/johng/gf/issues/INZS1](https://gitee.com/johng/gf/issues/INZS1));
1. 修复`gproc.ShellRun`在windows下的执行问题;
## 问题修复
1、修正gstr.IsNumeric错误
2、修复当xml中encoding字符集为非UTF-8字符集时报错的问题
3、修正gconv包float32->float64精度问题
4、修复gpage包分页计数问题
5、修复gdb批量数据Save错误
6、去掉gpool中math.MAXINT64常量的使用以修复int64到int类型的转换错误兼容32位系统
7、修正ghttp包没有使用Server仍然初始化相关异步goroutine的问题
# `v1.0.898 stable` (2018-10-24)
@ -227,3 +140,145 @@
1. 其他一些改动;
# `v0.99.682 beta` (2018-08-07)
## 新特性
1、新增gdes包用于DES加密/加密算法处理;
2、新增gkafka包kafka的golang客户端
3、新增gpool对象复用池比较于标准库的sync.Pool更加灵活强大可自定义对象的缓存时间、创建方法、销毁方法(http://gf.johng.cn/686654)
4、完成网络通信gtcp/gudp包的重构并进行了大量的改进工作新增了详尽的开发文档及示例代码(http://gf.johng.cn/494382)
5、增加gring并发安全环标准库container/ring包的并发安全版本并做了易用性的封装(http://gf.johng.cn/686655)
6、gtime包新增了自定义日期格式话的支持格式化语法类似PHP的date语法(http://gf.johng.cn/494387)
7、gdb增加调试模式特性使用SetDebug方法实现在调试模式下可以获取详细的SQL执行记录增加了详细的开发文档及示例代码(http://gf.johng.cn/702801)
8、gdb增加查询缓存特性使用Cache方法实现增加了详细的开发文档及示例代码(http://gf.johng.cn/702801)
9、ghttp.Server路由功能增加字段匹配规则特性支持如/order/list/{page}.html 动态路由规则特性(http://gf.johng.cn/702766)
10、gpage分页包增加分页URL规则生成模板特性内部可使用{.page}变量指定页码位置(http://gf.johng.cn/716438)
11、增加gmap.Map对象这是gmap.InterfaceInterfaceMap的别名
## 新功能
1、gdb增加MaxIdleConnCount/MaxOpenConnCount/MaxConnLifetime三项配置并增加SetMaxConnLifetime方法
2、ghttp.Client增加HTTP账号密码设置功能(SetBasicAuth)
3、glog新增对系统换行符号的自适配调整(\n|\r\n)
4、增加glog控制台调试模式打印开关(SetDebug)
5、gcfg增加SetFileName方法设置默认读取的配置文件名称
6、gcfg/gjson/gparser包新增Int8/16/32/64,Uint8/16/32/64方法
7、增加gzip方法的封装(Zip/Unzip)
8、gview增加模板变量分隔符设置方法SetDelimiters
9、ghttp.Response增加Writef、Writefln方法
## 功能改进
1、改进gfilepool文件指针池设计改进gfile文本内容写入增加指针池使用
2、gdb包增加调试模式特性并支持在调试模式下获得已执行的SQL列表结果
3、改进gproc进程间通信机制增加进程消息分组特性并限定队列大小
4、gdb结果方法处理增加ToXml/ToJson方法
5、gregx包名修改为gregex
6、改进gtime.StrToTime方法新增对常见标准时间日期的自动转换以及对时区的自动识别支持并调整gconv,gvalid对该包的引用
7、增加对字符集转换的封装gxml包中使用新增的字符集转换包来做处理
8、ghttp.Server.EnableAdmin页面Restart接口支持GET参数newExeFilePath支持
9、ghttp.Server平滑重启机制增加可自定义重启可执行文件路径特别是针对windows系统特别有用(因为windows下不支持可执行文件覆盖更新)
10、改进ghttp.Server静态文件检索设计增加开发环境时的main包源码目录查找机制改进gcfg/gview的main包源码目录查找机制
11、优化gcache设计LRU特性非默认开启优化gtype/gcache基准测试脚本新增gregx基准测试脚本改进设计提升性能
12、gfile包增加GoRootOfBuild方法用于获取编译时的GOROOT数值并改进glog包中backtrace的GOROOT路径过滤处理
13、改进grpool代码质量并改进对池化goroutine数量的限制设计
14、改进gdb.Map/List及g.Map/List的类型定义改用别名特性以便支持原生类型输入(map/slice)并修复gdb.Model.Update方法参数处理问题
15、调整ghttp包示例代码目录结构增加ghttp.Client自定义Header方法ghttp.Cookie增加Map方法用于获得客户端提交的所有cookie值构造成map返回
16、删除gcharset中的getcharset方法
17、去掉gmap中常用的基本数据类型转换获取方法
18、改进gconv.String方法当无法使用基本类型进行字符串转换时使用json.Marshal进行转换
19、gvalid.CheckObject方法名称修改为gvalid.CheckStruct
## 问题修复
1、修正gstr.IsNumeric错误
2、修复当xml中encoding字符集为非UTF-8字符集时报错的问题
3、修正gconv包float32->float64精度问题
4、修复gpage包分页计数问题
5、修复gdb批量数据Save错误
6、去掉gpool中math.MAXINT64常量的使用以修复int64到int类型的转换错误兼容32位系统
7、修正ghttp包没有使用Server仍然初始化相关异步goroutine的问题
# `v0.98.503 beta` (2018-05-21)
## 新特性
1、平滑重启特性( http://gf.johng.cn/625833 )
2、gflock文件锁模块( http://gf.johng.cn/626062 )
3、gproc进程管理及通信模块( http://gf.johng.cn/626063 )
4、gpage分页管理模块强大的动态分页及静态分页功能并为开发者自定义分页样式提供了极高的灵活度( http://gf.johng.cn/597431 )
5、ghttp.Server增加多端口监听特性并支持HTTP/HTTPS( http://gf.johng.cn/494366 , http://gf.johng.cn/598802 )
6、增加gspath目录检索包管理工具支持对多目录下的文件检索特性
7、ghttp包控制器及执行对象注册增加更灵活的动态路由特性路由规则增加{method}变量支持;
## 新功能
1、gutil包增加MapToStruct方法支持将map数据类型映射为struct对象
2、gconv
1)、gconv包增加按照类型名称字符串进行类型转换
2)、gconv包新增Time/TimeDuration类型转换方法
3、ghttp
1)、增加Web Server目录安全访问控制机制
2)、ghttp.Server增加自定义状态码回调函数注册处理
4、gdb
1)、gdb包增加gdb.GetStruct/gdb.Model.Struct方法获取查询结果记录自动转换为指定对象
2)、gdb增加Value/Record/Result类型增加对Value类型的系列类型转换方法
3)、gdb包增加db.GetCount,tx.GetCount,model.Count数量查询方法
## 功能改进
1、改进gredis客户端功能封装
2、改进grand包随机数生成性能
3、grand/gdb/gredis包增加benchmark性能测试脚本
4、改进gjson/gparser包的ToStruct方法实现
5、gdb 改进gdb.New获取ORM操作对象性能
6、gcfg :改进配置文件检索功能;
7、gview模板引擎增加多目录检索功能
8、gfile增加源码main包目录获取方法MainPkgPath
9、ghttp
1)、ghttp.Request增加请求进入和完成时间记录并增加到默认日志内容中
2)、ghttp.Server事件回调之间支持通过ghttp.Request.Param自定义参数进行流程传参
10、gdb
1)、改进gdb.Result与gdb.List, gdb.Record与gdb.Map之间的类型转换便于业务层数据编码处理(如json/xml)
2)、改进gdb.Tx.GetValue返回值类型
3)、gdb.Model.Data参数支持更加灵活的map参数
## 问题修复
1、ghttp
1)、修复ghttp包路由缓存问题
2)、修复服务注册时的控制器及执行对象方法丢失问题;
2、gconv
1)、修正gconv.Float64方法位大小设置问题
2)、修复gconv.Int64(float64(xxx))问题;
2、gdb
1)、修复gdb.GetAll针对返回数据列表的for..range...的返回结果slice相同指针问题
2)、修复gdb.Delete方法错误
3)、修复gdb.Model.And/Or方法
4)、修复gdb.Model.Where方法参数处理问题
3、garray修复garray包Remove方法锁机制问题
4、gtype 修复gtype.Float32/gtype.Float64对象类型的方法逻辑错误
5、gfsnotify修复在windows下文件参数中不同文件分隔符引起的热更新机制失效问题
6、修复gvalid包验证问题如果值为nil并且不需要require*验证时,其他验证失效。并增加单元测试项,测试通过。
# `v0.97.399 beta` (2018-04-23)
1、 增加gfsnotify文件监控模块
2、 配置管理模块增加配置文件自动检测更新机制;
3、 模板引擎增加对模板文件的自动检测更新机制;
4、 改进gconv包基本类型转换功能提高转换性能
5、 增加gpage分页管理包支持动态分页、静态分页以及自定义分页样式特性
6、 ghttp.Request增加Exit方法用以标记服务退出当在服务执行前调用后服务将不再执行
7、 ghttp.Response去掉WriteString方法统一使用Write方法返回数据流是使用灵活的参数形式
8、 模板引擎增加模板变量暴露接口LockFunc/RLockFunc以便支持开发者灵活处理模板变量
9、 ghttp.Server增加access & error log功能并支持开发者自定义日志处理回调函数注册
10、增加gredis包支持对redis的客户端操作封装并将gredis.Redis对象加入到gins单例管理器中进行统一配置管理维护
11、gins单例管理器增加对单例对象配置文件的自动检测更新机制当配置文件在外部发生变更时自动刷新单例管理器中的单例对象
12、gdb数据库ORM包增加And/Or条件链式方法并改进Where/Data方法参数灵活性
13、对于新增加的模块同时也增加了对应的开发文档并梳理完善了现有的其他模块开发文档
14、修复ISSUE:
#IISWI gitee.com/johng/gf/issues/IISWI,
#IISMY gitee.com/johng/gf/issues/IISMY,
反馈并跟踪完成第三方依赖mxj包的ISSUE修复(github.com/clbanning/mxj/issues/48)

12
TODO.MD
View File

@ -34,16 +34,16 @@
- glog分类&日志等级&链式操作、gdb debug自动输出调试信息、gmlock内存锁、
1. 服务注册域名增加对泛域名的支持;
1. Cookie设置中文失效问题
1. ghttp hook回调使用方式在注册路由比较多的时候优先级可能使得开发者混乱考虑方式便于管理
1. 使用gconv将slice映射到struct属性上例如redis hscan的结果集
1. 项目参考:
- https://github.com/namreg/godown
- https://github.com/Masterminds/sprig
1. gform参考 https://gohouse.github.io/gorose/dist/index.html 进行改进
1. 模板引擎增加对对象的支持(参考https://segmentfault.com/q/1010000016829214)
1. 改进gfpool在文件指针变化时的更新
1. gtcp提供简便的包发送/接收方法(SendPkg/RecvPkg)以解决常见的TCP通信粘包问题并完善文档参考https://www.cnblogs.com/kex1n/p/6502002.html
1. gfile对于文件的读写强行使用了gfpool在某些场景下不合适需要考虑剥离开并为开发者提供单独的指针池文件操作特性
1. 路由增加不区分大小写得匹配方式;
# DONE
@ -96,4 +96,8 @@
1. `gfsnotify`增加添加监听文件时的监听ID返回以便调用端删除监听时只删除自己添加的监听而不影响其他对该同一文件的监听回调
1. `gfsnotify`针对添加目录监听时无法使用多个`Watcher`,考虑改进,并考虑动态扩容全局`Watcher`方案;
1. 由于系统对inotify实例数量(`fs.inotify.max_user_instances`)以及队列大小(`fs.inotify.max_user_watches`)有限制,需要改进`gfsnotify`
1. WebServer事件回调允许对同一个路由规则绑定多个事件回调
1. gcfg/gview/ghttp等模块加上对临时文件目录的自动添加监听判断基本是开发环境下特别是windows环境去掉临时文件的监听避免临时文件过大引起的运行缓慢占用内存问题
1. 改进gfpool在文件指针变化时的更新
1. ghttp hook回调使用方式在注册路由比较多的时候优先级可能使得开发者混乱考虑方式便于管理
1. gform对于MySQL字段类型为datetime类型的时区问题分析

View File

@ -4,6 +4,7 @@
// If a copy of the MIT was not distributed with this file,
// You can obtain one at https://gitee.com/johng/gf.
// Package garray provides kinds of concurrent-safe(alternative) arrays.
// 并发安全的数组.
package garray

View File

@ -69,6 +69,17 @@ func (a *IntArray) InsertAfter(index int, value int) {
func (a *IntArray) Remove(index int) int {
a.mu.Lock()
defer a.mu.Unlock()
// 边界删除判断,以提高删除效率
if index == 0 {
value := a.array[0]
a.array = a.array[1 : ]
return value
} else if index == len(a.array) - 1 {
value := a.array[index]
a.array = a.array[: index]
return value
}
// 如果非边界删除,会涉及到数组创建,那么删除的效率差一些
value := a.array[index]
a.array = append(a.array[ : index], a.array[index + 1 : ]...)
return value
@ -136,6 +147,20 @@ func (a *IntArray) Search(value int) int {
return result
}
// 清理数组中重复的元素项
func (a *IntArray) Unique() *IntArray {
a.mu.Lock()
for i := 0; i < len(a.array) - 1; i++ {
for j := i + 1; j < len(a.array); j++ {
if a.array[i] == a.array[j] {
a.array = append(a.array[ : j], a.array[j + 1 : ]...)
}
}
}
a.mu.Unlock()
return a
}
// 使用自定义方法执行加锁修改操作
func (a *IntArray) LockFunc(f func(array []int)) {
a.mu.Lock(true)

View File

@ -68,11 +68,41 @@ func (a *Array) InsertAfter(index int, value interface{}) {
func (a *Array) Remove(index int) interface{} {
a.mu.Lock()
defer a.mu.Unlock()
// 边界删除判断,以提高删除效率
if index == 0 {
value := a.array[0]
a.array = a.array[1 : ]
return value
} else if index == len(a.array) - 1 {
value := a.array[index]
a.array = a.array[: index]
return value
}
// 如果非边界删除,会涉及到数组创建,那么删除的效率差一些
value := a.array[index]
a.array = append(a.array[ : index], a.array[index + 1 : ]...)
return value
}
// 将最左端(索引为0)的数据项移出数组,并返回该数据项
func (a *Array) PopLeft() interface{} {
a.mu.Lock()
defer a.mu.Unlock()
value := a.array[0]
a.array = a.array[1 : ]
return value
}
// 将最右端(索引为length - 1)的数据项移出数组,并返回该数据项
func (a *Array) PopRight() interface{} {
a.mu.Lock()
defer a.mu.Unlock()
index := len(a.array) - 1
value := a.array[index]
a.array = a.array[: index]
return value
}
// 追加数据项
func (a *Array) Append(value...interface{}) {
a.mu.Lock()
@ -135,6 +165,20 @@ func (a *Array) Search(value interface{}) int {
return result
}
// 清理数组中重复的元素项
func (a *Array) Unique() *Array {
a.mu.Lock()
for i := 0; i < len(a.array) - 1; i++ {
for j := i + 1; j < len(a.array); j++ {
if a.array[i] == a.array[j] {
a.array = append(a.array[ : j], a.array[j + 1 : ]...)
}
}
}
a.mu.Unlock()
return a
}
// 使用自定义方法执行加锁修改操作
func (a *Array) LockFunc(f func(array []interface{})) {
a.mu.Lock(true)

View File

@ -16,7 +16,7 @@ type SortedIntArray struct {
mu *rwmutex.RWMutex // 互斥锁
cap int // 初始化设置的数组容量
array []int // 底层数组
unique *gtype.Bool // 是否要求不能重复
unique *gtype.Bool // 是否要求不能重复(默认false)
compareFunc func(v1, v2 int) int // 比较函数,返回值 -1: v1 < v20: v1 == v21: v1 > v2
}
@ -161,7 +161,7 @@ func (a *SortedIntArray) binSearch(value int, lock bool) (index int, result int)
case 0 :
case 1 : min = mid + 1
}
if cmp == 0 || min > max {
if cmp == 0 || min >= max {
break
}
}

View File

@ -154,7 +154,7 @@ func (a *SortedArray) binSearch(value interface{}, lock bool)(index int, result
case 0 :
case 1 : min = mid + 1
}
if cmp == 0 || min > max {
if cmp == 0 || min >= max {
break
}
}

View File

@ -155,7 +155,7 @@ func (a *SortedStringArray) binSearch(value string, lock bool) (index int, resul
case 0 :
case 1 : min = mid + 1
}
if cmp == 0 || min > max {
if cmp == 0 || min >= max {
break
}
}

View File

@ -68,7 +68,18 @@ func (a *StringArray) InsertAfter(index int, value string) {
// 删除指定索引的数据项, 调用方注意判断数组边界
func (a *StringArray) Remove(index int) string {
a.mu.Lock()
defer a.mu.RUnlock()
defer a.mu.Unlock()
// 边界删除判断,以提高删除效率
if index == 0 {
value := a.array[0]
a.array = a.array[1 : ]
return value
} else if index == len(a.array) - 1 {
value := a.array[index]
a.array = a.array[: index]
return value
}
// 如果非边界删除,会涉及到数组创建,那么删除的效率差一些
value := a.array[index]
a.array = append(a.array[ : index], a.array[index + 1 : ]...)
return value
@ -135,6 +146,20 @@ func (a *StringArray) Search(value string) int {
return result
}
// 清理数组中重复的元素项
func (a *StringArray) Unique() *StringArray {
a.mu.Lock()
for i := 0; i < len(a.array) - 1; i++ {
for j := i + 1; j < len(a.array); j++ {
if a.array[i] == a.array[j] {
a.array = append(a.array[ : j], a.array[j + 1 : ]...)
}
}
}
a.mu.Unlock()
return a
}
// 使用自定义方法执行加锁修改操作
func (a *StringArray) LockFunc(f func(array []string)) {
a.mu.Lock(true)

View File

@ -0,0 +1,26 @@
// Copyright 2018 gf Author(https://gitee.com/johng/gf). All Rights Reserved.
//
// This Source Code Form is subject to the terms of the MIT License.
// If a copy of the MIT was not distributed with this file,
// You can obtain one at https://gitee.com/johng/gf.
// go test *.go
package garray_test
import (
"fmt"
"gitee.com/johng/gf/g/container/garray"
"testing"
)
func TestArray_Unique(t *testing.T) {
expect := []int{1, 2, 3, 4, 5, 6}
array := garray.NewIntArray(0, 0)
array.Append(1, 1, 2, 3, 3, 4, 4, 5, 5, 6, 6)
array.Unique()
if fmt.Sprint(array.Slice()) != fmt.Sprint(expect) {
t.Errorf("get: %v, expect: %v\n", array.Slice(), expect)
}
}

View File

@ -4,6 +4,7 @@
// If a copy of the MIT was not distributed with this file,
// You can obtain one at https://gitee.com/johng/gf.
// Package gchan provides graceful operations for channel.
// 优雅的Channel操作.
package gchan
@ -40,8 +41,7 @@ func (q *Chan) Pop() interface{} {
// 关闭队列(通知所有通过Pop阻塞的协程退出)
func (q *Chan) Close() {
if !q.closed.Val() {
q.closed.Set(true)
if !q.closed.Set(true) {
close(q.list)
}
}

View File

@ -5,6 +5,7 @@
// You can obtain one at https://gitee.com/johng/gf.
//
// Package glist provides a concurrent-safe(alternative) doubly linked list.
// 并发安全的双向链表.
package glist
@ -21,7 +22,7 @@ type List struct {
// 获得一个变长链表指针
func New(safe...bool) *List {
return &List{
return &List {
mu : rwmutex.New(safe...),
list : list.New(),
}

View File

@ -14,31 +14,31 @@ import (
var l = New()
func BenchmarkPushBack(b *testing.B) {
func Benchmark_PushBack(b *testing.B) {
for i := 0; i < b.N; i++ {
l.PushBack(i)
}
}
func BenchmarkPopFront(b *testing.B) {
func Benchmark_PopFront(b *testing.B) {
for i := 0; i < b.N; i++ {
l.PopFront()
}
}
func BenchmarkPushFront(b *testing.B) {
func Benchmark_PushFront(b *testing.B) {
for i := 0; i < b.N; i++ {
l.PushFront(i)
}
}
func BenchmarkPopBack(b *testing.B) {
func Benchmark_PopBack(b *testing.B) {
for i := 0; i < b.N; i++ {
l.PopBack()
}
}
func BenchmarkLen(b *testing.B) {
func Benchmark_Len(b *testing.B) {
for i := 0; i < b.N; i++ {
l.Len()
}

View File

@ -4,6 +4,7 @@
// If a copy of the MIT was not distributed with this file,
// You can obtain one at https://gitee.com/johng/gf.
// Package gmap provides kinds of concurrent-safe(alternative) maps.
// 并发安全的哈希MAP.
package gmap

View File

@ -4,6 +4,7 @@
// If a copy of the MIT was not distributed with this file,
// You can obtain one at https://gitee.com/johng/gf.
// Package gpool provides a object-reusable concurrent-safe pool.
// 对象复用池.
package gpool

View File

@ -4,6 +4,7 @@
// If a copy of the MIT was not distributed with this file,
// You can obtain one at https://gitee.com/johng/gf.
// Package gqueue provides a dynamic/static concurrent-safe(alternative) queue.
// 并发安全的动态队列.
// 特点:
// 1、动态队列初始化速度快
@ -29,26 +30,22 @@ type Queue struct {
}
const (
// 默认临时队列大小,注意是临时的
gDEFAULT_QUEUE_SIZE = 10000
// 动态队列缓冲区大小
gQUEUE_SIZE = 10000
)
// 队列大小为非必须参数,默认不限制
func New(limit...int) *Queue {
size := gDEFAULT_QUEUE_SIZE
if len(limit) > 0 {
size = limit[0]
}
q := &Queue {
list : glist.New(),
queue : make(chan interface{}, size),
events : make(chan struct{}, math.MaxInt32),
closeChan : make(chan struct{}, 0),
}
if len(limit) > 0 {
q.limit = size
q.limit = limit[0]
q.queue = make(chan interface{}, limit[0])
} else {
// 如果是动态队列大小那么额外会运行一个goroutine
q.list = glist.New()
q.queue = make(chan interface{}, gQUEUE_SIZE)
q.events = make(chan struct{}, math.MaxInt32)
go q.startAsyncLoop()
}
return q

View File

@ -13,36 +13,38 @@ import (
"gitee.com/johng/gf/g/container/gqueue"
)
var length = 10000000
var bn = 20000000
var length = 1000000
var qstatic = gqueue.New(length)
var qdynamic = gqueue.New()
var cany = make(chan interface{}, length)
var cint = make(chan int, length)
func Benchmark_GqueueStaticPushAndPop(b *testing.B) {
func Benchmark_Gqueue_StaticPushAndPop(b *testing.B) {
b.N = bn
for i := 0; i < b.N; i++ {
qstatic.Push(i)
qstatic.Pop()
}
}
func Benchmark_GqueueDynamicPush(b *testing.B) {
func Benchmark_Gqueue_DynamicPush(b *testing.B) {
b.N = bn
for i := 0; i < b.N; i++ {
qdynamic.Push(i)
}
}
func Benchmark_ChannelInterfacePushAndPop(b *testing.B) {
func Benchmark_Gqueue_DynamicPop(b *testing.B) {
b.N = bn
for i := 0; i < b.N; i++ {
qdynamic.Pop()
}
}
func Benchmark_Channel_PushAndPop(b *testing.B) {
b.N = bn
for i := 0; i < b.N; i++ {
cany <- i
<- cany
}
}
func Benchmark_ChannelIntPushAndPop(b *testing.B) {
for i := 0; i < b.N; i++ {
cint <- i
<- cint
}
}

View File

@ -4,6 +4,7 @@
// If a copy of the MIT was not distributed with this file,
// You can obtain one at https://gitee.com/johng/gf.
// Package gring provides a concurrent-safe(alternative) ring(circular lists).
// 并发安全的环.
package gring

View File

@ -4,6 +4,7 @@
// If a copy of the MIT was not distributed with this file,
// You can obtain one at https://gitee.com/johng/gf.
// Package gset provides kinds of concurrent-safe(alternative) sets.
// 并发安全的集合SET.
package gset

View File

@ -18,55 +18,55 @@ var ints = gset.NewIntSet()
var itfs = gset.NewInterfaceSet()
var strs = gset.NewStringSet()
func BenchmarkIntSet_Add(b *testing.B) {
func Benchmark_IntSet_Add(b *testing.B) {
for i := 0; i < b.N; i++ {
ints.Add(i)
}
}
func BenchmarkIntSet_Contains(b *testing.B) {
func Benchmark_IntSet_Contains(b *testing.B) {
for i := 0; i < b.N; i++ {
ints.Contains(i)
}
}
func BenchmarkIntSet_Remove(b *testing.B) {
func Benchmark_IntSet_Remove(b *testing.B) {
for i := 0; i < b.N; i++ {
ints.Remove(i)
}
}
func BenchmarkInterfaceSet_Add(b *testing.B) {
func Benchmark_InterfaceSet_Add(b *testing.B) {
for i := 0; i < b.N; i++ {
itfs.Add(i)
}
}
func BenchmarkInterfaceSet_Contains(b *testing.B) {
func Benchmark_InterfaceSet_Contains(b *testing.B) {
for i := 0; i < b.N; i++ {
itfs.Contains(i)
}
}
func BenchmarkInterfaceSet_Remove(b *testing.B) {
func Benchmark_InterfaceSet_Remove(b *testing.B) {
for i := 0; i < b.N; i++ {
itfs.Remove(i)
}
}
func BenchmarkStringSet_Add(b *testing.B) {
func Benchmark_StringSet_Add(b *testing.B) {
for i := 0; i < b.N; i++ {
strs.Add(strconv.Itoa(i))
}
}
func BenchmarkStringSet_Contains(b *testing.B) {
func Benchmark_StringSet_Contains(b *testing.B) {
for i := 0; i < b.N; i++ {
strs.Contains(strconv.Itoa(i))
}
}
func BenchmarkStringSet_Remove(b *testing.B) {
func Benchmark_StringSet_Remove(b *testing.B) {
for i := 0; i < b.N; i++ {
strs.Remove(strconv.Itoa(i))
}

View File

@ -0,0 +1,73 @@
// Copyright 2017 gf Author(https://gitee.com/johng/gf). All Rights Reserved.
//
// This Source Code Form is subject to the terms of the MIT License.
// If a copy of the MIT was not distributed with this file,
// You can obtain one at https://gitee.com/johng/gf.
// go test *.go -bench=".*"
package gset_test
import (
"testing"
"strconv"
"gitee.com/johng/gf/g/container/gset"
)
var intsUnsafe = gset.NewIntSet(false)
var itfsUnsafe = gset.NewInterfaceSet(false)
var strsUnsafe = gset.NewStringSet(false)
func Benchmark_Unsafe_IntSet_Add(b *testing.B) {
for i := 0; i < b.N; i++ {
intsUnsafe.Add(i)
}
}
func Benchmark_Unsafe_IntSet_Contains(b *testing.B) {
for i := 0; i < b.N; i++ {
intsUnsafe.Contains(i)
}
}
func Benchmark_Unsafe_IntSet_Remove(b *testing.B) {
for i := 0; i < b.N; i++ {
intsUnsafe.Remove(i)
}
}
func Benchmark_Unsafe_InterfaceSet_Add(b *testing.B) {
for i := 0; i < b.N; i++ {
itfsUnsafe.Add(i)
}
}
func Benchmark_Unsafe_InterfaceSet_Contains(b *testing.B) {
for i := 0; i < b.N; i++ {
itfsUnsafe.Contains(i)
}
}
func Benchmark_Unsafe_InterfaceSet_Remove(b *testing.B) {
for i := 0; i < b.N; i++ {
itfsUnsafe.Remove(i)
}
}
func Benchmark_Unsafe_StringSet_Add(b *testing.B) {
for i := 0; i < b.N; i++ {
strsUnsafe.Add(strconv.Itoa(i))
}
}
func Benchmark_Unsafe_StringSet_Contains(b *testing.B) {
for i := 0; i < b.N; i++ {
strsUnsafe.Contains(strconv.Itoa(i))
}
}
func Benchmark_Unsafe_StringSet_Remove(b *testing.B) {
for i := 0; i < b.N; i++ {
strsUnsafe.Remove(strconv.Itoa(i))
}
}

View File

@ -4,6 +4,7 @@
// If a copy of the MIT was not distributed with this file,
// You can obtain one at https://gitee.com/johng/gf.
// Package gtype provides kinds of concurrent-safe basic-types.
// 并发安全的基本类型.
package gtype

View File

@ -4,13 +4,14 @@
// If a copy of the MIT was not distributed with this file,
// You can obtain one at https://gitee.com/johng/gf.
// Package gvar provides a universal variable type.
// 通用动态变量.
package gvar
import (
"time"
"gitee.com/johng/gf/g/util/gconv"
"gitee.com/johng/gf/g/container/gtype"
"gitee.com/johng/gf/g/util/gconv"
"time"
)
type Var struct {
@ -30,6 +31,16 @@ func New(value interface{}, safe...bool) *Var {
return v
}
// 创建一个只读动态变量value参数可以为nil
func NewRead(value interface{}, safe...bool) VarRead {
return VarRead(New(value, safe...))
}
// 返回动态变量的只读接口
func (v *Var) ReadOnly() VarRead {
return VarRead(v)
}
func (v *Var) Set(value interface{}) (old interface{}) {
if v.safe {
old = v.value.(*gtype.Interface).Set(value)
@ -48,6 +59,7 @@ func (v *Var) Val() interface{} {
}
}
// Val() 别名
func (v *Var) Interface() interface{} {
return v.Val()
}

View File

@ -0,0 +1,38 @@
// Copyright 2018 gf Author(https://gitee.com/johng/gf). All Rights Reserved.
//
// This Source Code Form is subject to the terms of the MIT License.
// If a copy of the MIT was not distributed with this file,
// You can obtain one at https://gitee.com/johng/gf.
package gvar
import "time"
// 只读变量接口
type VarRead interface {
Val() interface{}
IsNil() bool
Bytes() []byte
String() string
Bool() bool
Int() int
Int8() int8
Int16() int16
Int32() int32
Int64() int64
Uint() uint
Uint8() uint8
Uint16() uint16
Uint32() uint32
Uint64() uint64
Float32() float32
Float64() float64
Interface() interface{}
Ints() []int
Floats() []float64
Strings() []string
Interfaces() []interface{}
Time(format ...string) time.Time
TimeDuration() time.Duration
Struct(objPointer interface{}, attrMapping ...map[string]string) error
}

View File

@ -91,7 +91,7 @@ type Db struct {
charr string // SQL安全符号(右)
debug *gtype.Bool // (默认关闭)是否开启调试模式,当开启时会启用一些调试特性
sqls *gring.Ring // (debug=true时有效)已执行的SQL列表
cache *gcache.Cache // 查询缓存需要注意的是,事务查询不支持缓存
cache *gcache.Cache // 数据库缓存,包括底层连接池对象缓存及查询缓存需要注意的是,事务查询不支持查询缓存
maxIdleConnCount *gtype.Int // 连接池最大限制的连接数
maxOpenConnCount *gtype.Int // 连接池最大打开的连接数
maxConnLifetime *gtype.Int // (单位秒)连接对象可重复使用的时间长度
@ -108,7 +108,7 @@ type Sql struct {
}
// 返回数据表记录值
type Value = *gvar.Var
type Value = gvar.VarRead
// 返回数据表记录Map
type Record map[string]Value
@ -133,6 +133,7 @@ func init() {
driverMap["oracle"] = linkOracle
driverMap["sqlite"] = linkSqlite
driverMap["pgsql"] = linkPgsql
driverMap["mssql"] = linkMssql
}
// 使用默认/指定分组配置进行连接数据库集群配置项default
@ -247,40 +248,51 @@ func getLinkByType(dbType string) (Link, error) {
}
// 获得底层数据库链接对象
func (db *Db) getSqlDb(master bool) (*sql.DB, error) {
func (db *Db) getSqlDb(master bool) (sqlDb *sql.DB, err error) {
// 负载均衡
node, err := getConfigNodeByGroup(db.group, master)
if err != nil {
return nil, err
}
// 类型对象
link, err := getLinkByType(node.Type)
if err != nil {
return nil, err
}
sqlDb, err := link.Open(node)
if err != nil {
return nil, err
}
if node.MaxIdleConnCount > 0 {
sqlDb.SetMaxIdleConns(node.MaxIdleConnCount)
}
if n := db.maxIdleConnCount.Val(); n > 0 {
sqlDb.SetMaxIdleConns(n)
// 检查缓存连接池对象
cacheKey := node.String()
if v := db.cache.Get(cacheKey); v != nil {
return v.(*sql.DB), nil
}
v := db.cache.GetOrSetFuncLock(node.String(), func() interface{} {
sqlDb, err = link.Open(node)
if err != nil {
return nil
}
if node.MaxOpenConnCount > 0 {
sqlDb.SetMaxOpenConns(node.MaxOpenConnCount)
}
if n := db.maxOpenConnCount.Val(); n > 0 {
sqlDb.SetMaxOpenConns(n)
}
if n := db.maxIdleConnCount.Val(); n > 0 {
sqlDb.SetMaxIdleConns(n)
} else if node.MaxIdleConnCount > 0 {
sqlDb.SetMaxIdleConns(node.MaxIdleConnCount)
}
if node.MaxConnLifetime > 0 {
sqlDb.SetConnMaxLifetime(time.Duration(node.MaxConnLifetime) * time.Second)
if n := db.maxOpenConnCount.Val(); n > 0 {
sqlDb.SetMaxOpenConns(n)
} else if node.MaxOpenConnCount > 0 {
sqlDb.SetMaxOpenConns(node.MaxOpenConnCount)
}
if n := db.maxConnLifetime.Val(); n > 0 {
sqlDb.SetConnMaxLifetime(time.Duration(n) * time.Second)
} else if node.MaxConnLifetime > 0 {
sqlDb.SetConnMaxLifetime(time.Duration(node.MaxConnLifetime) * time.Second)
}
return sqlDb
}, 0)
if v != nil && sqlDb == nil {
sqlDb = v.(*sql.DB)
}
if n := db.maxConnLifetime.Val(); n > 0 {
sqlDb.SetConnMaxLifetime(time.Duration(n) * time.Second)
}
return sqlDb, nil
return
}
// 创建底层数据库master链接对象

View File

@ -89,7 +89,6 @@ func (db *Db) Query(query string, args ...interface{}) (*sql.Rows, error) {
if err != nil {
return nil,err
}
defer slave.Close()
p := db.link.handleSqlBeforeExec(&query)
if db.debug.Val() {
militime1 := gtime.Millisecond()
@ -125,7 +124,6 @@ func (db *Db) Exec(query string, args ...interface{}) (sql.Result, error) {
if err != nil {
return nil,err
}
defer master.Close()
p := db.link.handleSqlBeforeExec(&query)
if db.debug.Val() {
militime1 := gtime.Millisecond()
@ -190,7 +188,7 @@ func (db *Db) GetAll(query string, args ...interface{}) (Result, error) {
for i, col := range values {
v := make([]byte, len(col))
copy(v, col)
row[columns[i]] = gvar.New(v)
row[columns[i]] = gvar.New(v, false)
}
records = append(records, row)
}
@ -259,12 +257,10 @@ func (db *Db) Select(tables, fields string, condition interface{}, groupBy, orde
}
// sql预处理执行完成后调用返回值sql.Stmt.Exec完成sql操作
// 记得调用sql.Stmt.Close关闭操作对象
func (db *Db) Prepare(query string) (*sql.Stmt, error) {
if master, err := db.Master(); err != nil {
return nil, err
} else {
defer master.Close()
return master.Prepare(query)
}
}
@ -274,7 +270,6 @@ func (db *Db) PingMaster() error {
if master, err := db.Master(); err != nil {
return err
} else {
defer master.Close()
return master.Ping()
}
}
@ -284,7 +279,6 @@ func (db *Db) PingSlave() error {
if slave, err := db.Slave(); err != nil {
return err
} else {
defer slave.Close()
return slave.Ping()
}
}

View File

@ -8,6 +8,7 @@
package gdb
import (
"fmt"
"sync"
)
@ -134,4 +135,15 @@ func (db *Db) SetMaxOpenConns(n int) {
// 如果 d <= 0 表示该链接会一直重复利用
func (db *Db) SetConnMaxLifetime(n int) {
db.maxConnLifetime.Set(n)
}
// 节点配置转换为字符串
func (node *ConfigNode) String() string {
if node.Linkinfo != "" {
return node.Linkinfo
}
return fmt.Sprintf(`%s@%s:%s,%s,%s,%s,%s,%d-%d-%d`, node.User, node.Host, node.Port,
node.Name, node.Type, node.Role, node.Charset,
node.MaxIdleConnCount, node.MaxOpenConnCount, node.MaxConnLifetime,
)
}

View File

@ -344,6 +344,8 @@ func (md *Model) Struct(obj interface{}) error {
func (md *Model) Count() (int, error) {
if md.fields == "" || md.fields == "*" {
md.fields = "COUNT(1)"
} else {
md.fields = fmt.Sprintf(`COUNT(%s)`, md.fields)
}
s := md.getFormattedSql()
if len(md.groupBy) > 0 {

159
g/database/gdb/gdb_mssql.go Normal file
View File

@ -0,0 +1,159 @@
// Copyright 2017 gf Author(https://gitee.com/johng/gf). All Rights Reserved.
//
// This Source Code Form is subject to the terms of the MIT License.
// If a copy of the MIT was not distributed with this file,
// You can obtain one at https://gitee.com/johng/gf.
/*
@author wenzi1<liyz23@qq.com>
@date 20181109
说明:
1.需要导入sqlserver驱动 github.com/denisenkom/go-mssqldb
2.不支持save/replace方法
3.不支持LastInsertId方法
*/
package gdb
import (
"database/sql"
"fmt"
"gitee.com/johng/gf/g/util/gregex"
"strconv"
"strings"
)
var linkMssql = &dbmssql{}
// 数据库链接对象
type dbmssql struct {
Db
}
// 创建SQL操作对象
func (db *dbmssql) Open(c *ConfigNode) (*sql.DB, error) {
var source string
if c.Linkinfo != "" {
source = c.Linkinfo
} else {
source = fmt.Sprintf("user id=%s;password=%s;server=%s;port=%s;database=%s;encrypt=disable", c.User, c.Pass, c.Host, c.Port, c.Name)
}
if db, err := sql.Open("sqlserver", source); err == nil {
return db, nil
} else {
return nil, err
}
}
// 获得关键字操作符 - 左
func (db *dbmssql) getQuoteCharLeft() string {
return "\""
}
// 获得关键字操作符 - 右
func (db *dbmssql) getQuoteCharRight() string {
return "\""
}
// 在执行sql之前对sql进行进一步处理
func (db *dbmssql) handleSqlBeforeExec(q *string) *string {
index := 0
str, _ := gregex.ReplaceStringFunc("\\?", *q, func(s string) string {
index++
return fmt.Sprintf("@p%d", index)
})
str, _ = gregex.ReplaceString("\"", "", str)
return db.parseSql(&str)
}
//将MYSQL的SQL语法转换为MSSQL的语法
//1.由于mssql不支持limit写法所以需要对mysql中的limit用法做转换
func (db *dbmssql) parseSql(sql *string) *string {
//下面的正则表达式匹配出SELECT和INSERT的关键字后分别做不同的处理如有LIMIT则将LIMIT的关键字也匹配出
patten := `^\s*(?i)(SELECT)|(LIMIT\s*(\d+)\s*,\s*(\d+))`
if gregex.IsMatchString(patten, *sql) == false {
fmt.Println("not matched..")
return sql
}
res, err := gregex.MatchAllString(patten, *sql)
if err != nil {
fmt.Println("MatchString error.", err)
return nil
}
index := 0
keyword := strings.TrimSpace(res[index][0])
keyword = strings.ToUpper(keyword)
index++
switch keyword {
case "SELECT":
//不含LIMIT关键字则不处理
if len(res) < 2 || (strings.HasPrefix(res[index][0], "LIMIT") == false && strings.HasPrefix(res[index][0], "limit") == false) {
break
}
//不含LIMIT则不处理
if gregex.IsMatchString("((?i)SELECT)(.+)((?i)LIMIT)", *sql) == false {
break
}
//判断SQL中是否含有order by
selectStr := ""
orderbyStr := ""
haveOrderby := gregex.IsMatchString("((?i)SELECT)(.+)((?i)ORDER BY)", *sql)
if haveOrderby {
//取order by 前面的字符串
queryExpr, _ := gregex.MatchString("((?i)SELECT)(.+)((?i)ORDER BY)", *sql)
if len(queryExpr) != 4 || strings.EqualFold(queryExpr[1], "SELECT") == false || strings.EqualFold(queryExpr[3], "ORDER BY") == false{
break
}
selectStr = queryExpr[2]
//取order by表达式的值
orderbyExpr, _ := gregex.MatchString("((?i)ORDER BY)(.+)((?i)LIMIT)", *sql)
if len(orderbyExpr) != 4 || strings.EqualFold(orderbyExpr[1], "ORDER BY") == false || strings.EqualFold(orderbyExpr[3], "LIMIT") == false{
break
}
orderbyStr = orderbyExpr[2]
} else {
queryExpr, _ := gregex.MatchString("((?i)SELECT)(.+)((?i)LIMIT)", *sql)
if len(queryExpr) != 4 || strings.EqualFold(queryExpr[1], "SELECT") == false || strings.EqualFold(queryExpr[3], "LIMIT") == false{
break
}
selectStr = queryExpr[2]
}
//取limit后面的取值范围
first, limit := 0, 0
for i := 1; i < len(res[index]); i++ {
if len(strings.TrimSpace(res[index][i])) == 0 {
continue
}
if strings.HasPrefix(res[index][i], "LIMIT") || strings.HasPrefix(res[index][i], "limit") {
first, _ = strconv.Atoi(res[index][i+1])
limit, _ = strconv.Atoi(res[index][i+2])
break
}
}
if haveOrderby {
*sql = fmt.Sprintf("SELECT * FROM (SELECT ROW_NUMBER() OVER (ORDER BY %s) as ROWNUMBER_, %s ) as TMP_ WHERE TMP_.ROWNUMBER_ > %d AND TMP_.ROWNUMBER_ <= %d", orderbyStr, selectStr, first, limit)
} else {
if first == 0 {
first = limit
} else {
first = limit - first
}
*sql = fmt.Sprintf("SELECT * FROM (SELECT TOP %d * FROM (SELECT TOP %d %s) as TMP1_ ) as TMP2_ ", first, limit, selectStr)
}
default:
}
return sql
}

View File

@ -99,8 +99,9 @@ func (db *dboracle) parseSql(sql *string) *string {
}
queryExpr, _ := gregex.MatchString("((?i)SELECT)(.+)((?i)LIMIT)", *sql)
queryExpr[0] = strings.TrimRight(queryExpr[0], "LIMIT")
queryExpr[0] = strings.TrimRight(queryExpr[0], "limit")
if len(queryExpr) != 4 || strings.EqualFold(queryExpr[1], "SELECT") == false || strings.EqualFold(queryExpr[3], "LIMIT") == false{
break
}
//取limit后面的取值范围
first, limit := 0, 0
@ -117,7 +118,7 @@ func (db *dboracle) parseSql(sql *string) *string {
}
//也可以使用between,据说这种写法的性能会比between好点,里层SQL中的ROWNUM_ >= limit可以缩小查询后的数据集规模
*sql = fmt.Sprintf("SELECT * FROM (SELECT GFORM.*, ROWNUM ROWNUM_ FROM (%s) GFORM WHERE ROWNUM <= %d) WHERE ROWNUM_ >= %d", queryExpr[0], limit, first)
*sql = fmt.Sprintf("SELECT * FROM (SELECT GFORM.*, ROWNUM ROWNUM_ FROM (%s %s) GFORM WHERE ROWNUM <= %d) WHERE ROWNUM_ >= %d", queryExpr[1], queryExpr[2], limit, first)
case "INSERT":
//获取VALUE的值匹配所有带括号的值,会将INSERT INTO后的值匹配到所以下面的判断语句会判断数组长度是否小于3
valueExpr, err := gregex.MatchAllString(`(\s*\(([^\(\)]*)\))`, *sql)

View File

@ -27,16 +27,12 @@ type Tx struct {
// 事务操作,提交
func (tx *Tx) Commit() error {
err := tx.tx.Commit()
tx.master.Close()
return err
return tx.tx.Commit()
}
// 事务操作,回滚
func (tx *Tx) Rollback() error {
err := tx.tx.Rollback()
tx.master.Close()
return err
return tx.tx.Rollback()
}
// (事务)数据库sql查询操作主要执行查询
@ -123,7 +119,7 @@ func (tx *Tx) GetAll(query string, args ...interface{}) (Result, error) {
for i, col := range values {
v := make([]byte, len(col))
copy(v, col)
row[columns[i]] = gvar.New(v)
row[columns[i]] = gvar.New(v, false)
}
//fmt.Printf("%p\n", row["typeid"])
records = append(records, row)
@ -193,7 +189,6 @@ func (tx *Tx) Select(tables, fields string, condition interface{}, groupBy, orde
}
// sql预处理执行完成后调用返回值sql.Stmt.Exec完成sql操作
// 记得调用sql.Stmt.Close关闭操作对象
func (tx *Tx) Prepare(query string) (*sql.Stmt, error) {
return tx.tx.Prepare(query)
}

View File

@ -87,6 +87,12 @@ func (r *Redis) Close() error {
return r.pool.Close()
}
// 获得一个原生的redis连接对象用于自定义连接操作
// 但是需要注意的是如果不再使用该连接对象时需要手动Close连接否则会造成连接数超限。
func (r *Redis) GetConn() redis.Conn {
return r.pool.Get()
}
// 设置属性 - MaxIdle
func (r *Redis) SetMaxIdle(value int) {
r.pool.MaxIdle = value

View File

@ -174,24 +174,28 @@ func EncodeFloat64(f float64) []byte {
// 当b位数不够时进行高位补0
func fillUpSize(b []byte, l int) []byte {
if len(b) >= l {
return b
}
c := make([]byte, 0)
c = append(c, b...)
for i := 0; i <= l - len(b); i++ {
for i := 0; i < l - len(b); i++ {
c = append(c, 0x00)
}
return c
}
// 将二进制解析为int类型根据[]byte的长度进行自动转换
// 将二进制解析为int类型根据[]byte的长度进行自动转换.
// 注意内部使用的是uint*使用int会造成位丢失。
func DecodeToInt(b []byte) int {
if len(b) < 2 {
return int(DecodeToInt8(b))
return int(DecodeToUint8(b))
} else if len(b) < 3 {
return int(DecodeToInt16(b))
return int(DecodeToUint16(b))
} else if len(b) < 5 {
return int(DecodeToInt32(b))
return int(DecodeToUint32(b))
} else {
return int(DecodeToInt64(b))
return int(DecodeToUint64(b))
}
}

View File

@ -9,103 +9,106 @@
package ghash_test
import (
"testing"
"gitee.com/johng/gf/g/encoding/ghash"
"gitee.com/johng/gf/g/encoding/gbinary"
"testing"
)
var (
str = []byte("This is the test string for hash.")
)
func BenchmarkBKDRHash(b *testing.B) {
for i := 0; i < b.N; i++ {
ghash.BKDRHash(gbinary.EncodeInt(i))
ghash.BKDRHash(str)
}
}
func BenchmarkBKDRHash64(b *testing.B) {
for i := 0; i < b.N; i++ {
ghash.BKDRHash64(gbinary.EncodeInt(i))
ghash.BKDRHash64(str)
}
}
func BenchmarkSDBMHash(b *testing.B) {
for i := 0; i < b.N; i++ {
ghash.SDBMHash(gbinary.EncodeInt(i))
ghash.SDBMHash(str)
}
}
func BenchmarkSDBMHash64(b *testing.B) {
for i := 0; i < b.N; i++ {
ghash.SDBMHash64(gbinary.EncodeInt(i))
ghash.SDBMHash64(str)
}
}
func BenchmarkRSHash(b *testing.B) {
for i := 0; i < b.N; i++ {
ghash.RSHash(gbinary.EncodeInt(i))
ghash.RSHash(str)
}
}
func BenchmarkSRSHash64(b *testing.B) {
for i := 0; i < b.N; i++ {
ghash.RSHash64(gbinary.EncodeInt(i))
ghash.RSHash64(str)
}
}
func BenchmarkJSHash(b *testing.B) {
for i := 0; i < b.N; i++ {
ghash.JSHash(gbinary.EncodeInt(i))
ghash.JSHash(str)
}
}
func BenchmarkJSHash64(b *testing.B) {
for i := 0; i < b.N; i++ {
ghash.JSHash64(gbinary.EncodeInt(i))
ghash.JSHash64(str)
}
}
func BenchmarkPJWHash(b *testing.B) {
for i := 0; i < b.N; i++ {
ghash.PJWHash(gbinary.EncodeInt(i))
ghash.PJWHash(str)
}
}
func BenchmarkPJWHash64(b *testing.B) {
for i := 0; i < b.N; i++ {
ghash.PJWHash64(gbinary.EncodeInt(i))
ghash.PJWHash64(str)
}
}
func BenchmarkELFHash(b *testing.B) {
for i := 0; i < b.N; i++ {
ghash.ELFHash(gbinary.EncodeInt(i))
ghash.ELFHash(str)
}
}
func BenchmarkELFHash64(b *testing.B) {
for i := 0; i < b.N; i++ {
ghash.ELFHash64(gbinary.EncodeInt(i))
ghash.ELFHash64(str)
}
}
func BenchmarkDJBHash(b *testing.B) {
for i := 0; i < b.N; i++ {
ghash.DJBHash(gbinary.EncodeInt(i))
ghash.DJBHash(str)
}
}
func BenchmarkDJBHash64(b *testing.B) {
for i := 0; i < b.N; i++ {
ghash.DJBHash64(gbinary.EncodeInt(i))
ghash.DJBHash64(str)
}
}
func BenchmarkAPHash(b *testing.B) {
for i := 0; i < b.N; i++ {
ghash.APHash(gbinary.EncodeInt(i))
ghash.APHash(str)
}
}
func BenchmarkAPHash64(b *testing.B) {
for i := 0; i < b.N; i++ {
ghash.APHash64(gbinary.EncodeInt(i))
ghash.APHash64(str)
}
}

View File

@ -41,21 +41,29 @@ type Json struct {
func New(value interface{}, safe...bool) *Json {
j := (*Json)(nil)
switch value.(type) {
case map[string]interface{}, []interface{}, nil:
j = &Json{
p : &value,
c : byte(gDEFAULT_SPLIT_CHAR),
vc : false ,
}
default:
// 这里效率会比较低
b, _ := Encode(value)
v, _ := Decode(b)
j = &Json{
case map[string]interface{}, []interface{}, nil:
j = &Json{
p : &value,
c : byte(gDEFAULT_SPLIT_CHAR),
vc : false ,
}
default:
v := (interface{})(nil)
if m := gconv.Map(value); m != nil {
v = m
j = &Json {
p : &v,
c : byte(gDEFAULT_SPLIT_CHAR),
vc : false,
}
} else {
v = gconv.Interfaces(value)
j = &Json {
p : &v,
c : byte(gDEFAULT_SPLIT_CHAR),
vc : false,
}
}
}
j.mu = rwmutex.New(safe...)
return j
@ -116,27 +124,27 @@ func LoadContent (data []byte, dataType...string) (*Json, error) {
t = dataType[0]
}
switch t {
case "xml": fallthrough
case ".xml":
data, err = gxml.ToJson(data)
if err != nil {
return nil, err
}
case "yml": fallthrough
case "yaml": fallthrough
case ".yml": fallthrough
case ".yaml":
data, err = gyaml.ToJson(data)
if err != nil {
return nil, err
}
case "xml": fallthrough
case ".xml":
data, err = gxml.ToJson(data)
if err != nil {
return nil, err
}
case "yml": fallthrough
case "yaml": fallthrough
case ".yml": fallthrough
case ".yaml":
data, err = gyaml.ToJson(data)
if err != nil {
return nil, err
}
case "toml": fallthrough
case ".toml":
data, err = gtoml.ToJson(data)
if err != nil {
return nil, err
}
case "toml": fallthrough
case ".toml":
data, err = gtoml.ToJson(data)
if err != nil {
return nil, err
}
}
if err := json.Unmarshal(data, &result); err != nil {
return nil, err
@ -328,136 +336,136 @@ func (j *Json) setValue(pattern string, value interface{}, removed bool) error {
defer j.mu.Unlock()
for i:= 0; i < length; i++ {
switch (*pointer).(type) {
case map[string]interface{}:
if i == length - 1 {
case map[string]interface{}:
if i == length - 1 {
if removed && value == nil {
// 删除map元素
delete((*pointer).(map[string]interface{}), array[i])
} else {
(*pointer).(map[string]interface{})[array[i]] = value
}
} else {
// 当键名不存在的情况这里会进行处理
if v, ok := (*pointer).(map[string]interface{})[array[i]]; !ok {
if removed && value == nil {
// 删除map元素
delete((*pointer).(map[string]interface{}), array[i])
} else {
(*pointer).(map[string]interface{})[array[i]] = value
goto done
}
} else {
// 当键名不存在的情况这里会进行处理
if v, ok := (*pointer).(map[string]interface{})[array[i]]; !ok {
if removed && value == nil {
goto done
}
// 创建新节点
if gstr.IsNumeric(array[i + 1]) {
// 创建array节点
n, _ := strconv.Atoi(array[i + 1])
var v interface{} = make([]interface{}, n + 1)
pparent = j.setPointerWithValue(pointer, array[i], v)
pointer = &v
} else {
// 创建map节点
var v interface{} = make(map[string]interface{})
pparent = j.setPointerWithValue(pointer, array[i], v)
pointer = &v
}
} else {
pparent = pointer
pointer = &v
}
}
case []interface{}:
// 键名与当前指针类型不符合,需要执行**覆盖操作**
if !gstr.IsNumeric(array[i]) {
if i == length - 1 {
*pointer = map[string]interface{}{ array[i] : value }
} else {
var v interface{} = make(map[string]interface{})
*pointer = v
pparent = pointer
pointer = &v
}
continue
}
valn, err := strconv.Atoi(array[i])
if err != nil {
return err
}
// 叶子节点
if i == length - 1 {
if len((*pointer).([]interface{})) > valn {
if removed && value == nil {
// 删除数据元素
j.setPointerWithValue(pparent, array[i - 1], append((*pointer).([]interface{})[ : valn], (*pointer).([]interface{})[valn + 1 : ]...))
} else {
(*pointer).([]interface{})[valn] = value
}
} else {
if removed && value == nil {
goto done
}
if pparent == nil {
// 表示根节点
j.setPointerWithValue(pointer, array[i], value)
} else {
// 非根节点
s := make([]interface{}, valn + 1)
copy(s, (*pointer).([]interface{}))
s[valn] = value
j.setPointerWithValue(pparent, array[i - 1], s)
}
}
} else {
// 创建新节点
if gstr.IsNumeric(array[i + 1]) {
// 创建array节点
n, _ := strconv.Atoi(array[i + 1])
if len((*pointer).([]interface{})) > valn {
(*pointer).([]interface{})[valn] = make([]interface{}, n + 1)
pparent = pointer
pointer = &(*pointer).([]interface{})[valn]
} else {
if removed && value == nil {
goto done
}
var v interface{} = make([]interface{}, n + 1)
pparent = j.setPointerWithValue(pointer, array[i], v)
pointer = &v
}
var v interface{} = make([]interface{}, n + 1)
pparent = j.setPointerWithValue(pointer, array[i], v)
pointer = &v
} else {
// 创建map节点
var v interface{} = make(map[string]interface{})
pparent = j.setPointerWithValue(pointer, array[i], v)
pointer = &v
}
} else {
pparent = pointer
pointer = &v
}
}
// 如果当前指针指向的变量不是引用类型的,
// 那么修改变量必须通过父级进行修改,即 pparent
default:
if removed && value == nil {
goto done
case []interface{}:
// 键名与当前指针类型不符合,需要执行**覆盖操作**
if !gstr.IsNumeric(array[i]) {
if i == length - 1 {
*pointer = map[string]interface{}{ array[i] : value }
} else {
var v interface{} = make(map[string]interface{})
*pointer = v
pparent = pointer
pointer = &v
}
if gstr.IsNumeric(array[i]) {
n, _ := strconv.Atoi(array[i])
s := make([]interface{}, n + 1)
if i == length - 1 {
s[n] = value
}
if pparent != nil {
pparent = j.setPointerWithValue(pparent, array[i - 1], s)
continue
}
valn, err := strconv.Atoi(array[i])
if err != nil {
return err
}
// 叶子节点
if i == length - 1 {
if len((*pointer).([]interface{})) > valn {
if removed && value == nil {
// 删除数据元素
j.setPointerWithValue(pparent, array[i - 1], append((*pointer).([]interface{})[ : valn], (*pointer).([]interface{})[valn + 1 : ]...))
} else {
*pointer = s
pparent = pointer
(*pointer).([]interface{})[valn] = value
}
} else {
if removed && value == nil {
goto done
}
if pparent == nil {
// 表示根节点
j.setPointerWithValue(pointer, array[i], value)
} else {
// 非根节点
s := make([]interface{}, valn + 1)
copy(s, (*pointer).([]interface{}))
s[valn] = value
j.setPointerWithValue(pparent, array[i - 1], s)
}
}
} else {
if gstr.IsNumeric(array[i + 1]) {
n, _ := strconv.Atoi(array[i + 1])
if len((*pointer).([]interface{})) > valn {
(*pointer).([]interface{})[valn] = make([]interface{}, n + 1)
pparent = pointer
pointer = &(*pointer).([]interface{})[valn]
} else {
if removed && value == nil {
goto done
}
var v interface{} = make([]interface{}, n + 1)
pparent = j.setPointerWithValue(pointer, array[i], v)
pointer = &v
}
} else {
var v interface{} = make(map[string]interface{})
if i == length - 1 {
v = map[string]interface{}{
array[i] : value,
}
}
if pparent != nil {
pparent = j.setPointerWithValue(pparent, array[i - 1], v)
} else {
*pointer = v
pparent = pointer
}
pparent = j.setPointerWithValue(pointer, array[i], v)
pointer = &v
}
}
// 如果当前指针指向的变量不是引用类型的,
// 那么修改变量必须通过父级进行修改,即 pparent
default:
if removed && value == nil {
goto done
}
if gstr.IsNumeric(array[i]) {
n, _ := strconv.Atoi(array[i])
s := make([]interface{}, n + 1)
if i == length - 1 {
s[n] = value
}
if pparent != nil {
pparent = j.setPointerWithValue(pparent, array[i - 1], s)
} else {
*pointer = s
pparent = pointer
}
} else {
var v interface{} = make(map[string]interface{})
if i == length - 1 {
v = map[string]interface{}{
array[i] : value,
}
}
if pparent != nil {
pparent = j.setPointerWithValue(pparent, array[i - 1], v)
} else {
*pointer = v
pparent = pointer
}
pointer = &v
}
}
}
@ -468,16 +476,16 @@ done:
// 数据结构转换map参数必须转换为map[string]interface{},数组参数必须转换为[]interface{}
func (j *Json) convertValue(value interface{}) interface{} {
switch value.(type) {
case map[string]interface{}:
return value
case []interface{}:
return value
default:
// 这里效率会比较低,当然比直接用反射也不会差到哪儿去
// 为了操作的灵活性,牺牲了一定的效率
b, _ := Encode(value)
v, _ := Decode(b)
return v
case map[string]interface{}:
return value
case []interface{}:
return value
default:
// 这里效率会比较低,当然比直接用反射也不会差到哪儿去
// 为了操作的灵活性,牺牲了一定的效率
b, _ := Encode(value)
v, _ := Decode(b)
return v
}
return value
}
@ -486,23 +494,23 @@ func (j *Json) convertValue(value interface{}) interface{} {
// 返回修改后的父级指针
func (j *Json) setPointerWithValue(pointer *interface{}, key string, value interface{}) *interface{} {
switch (*pointer).(type) {
case map[string]interface{}:
(*pointer).(map[string]interface{})[key] = value
return &value
case []interface{}:
n, _ := strconv.Atoi(key)
if len((*pointer).([]interface{})) > n {
(*pointer).([]interface{})[n] = value
return &(*pointer).([]interface{})[n]
} else {
s := make([]interface{}, n + 1)
copy(s, (*pointer).([]interface{}))
s[n] = value
*pointer = s
return &s[n]
}
default:
*pointer = value
case map[string]interface{}:
(*pointer).(map[string]interface{})[key] = value
return &value
case []interface{}:
n, _ := strconv.Atoi(key)
if len((*pointer).([]interface{})) > n {
(*pointer).([]interface{})[n] = value
return &(*pointer).([]interface{})[n]
} else {
s := make([]interface{}, n + 1)
copy(s, (*pointer).([]interface{}))
s[n] = value
*pointer = s
return &s[n]
}
default:
*pointer = value
}
return pointer
}
@ -535,12 +543,12 @@ func (j *Json) Len(pattern string) int {
p := j.getPointerByPattern(pattern)
if p != nil {
switch (*p).(type) {
case map[string]interface{}:
return len((*p).(map[string]interface{}))
case []interface{}:
return len((*p).([]interface{}))
default:
return -1
case map[string]interface{}:
return len((*p).(map[string]interface{}))
case []interface{}:
return len((*p).([]interface{}))
default:
return -1
}
}
return -1
@ -627,17 +635,17 @@ func (j *Json) getPointerByPatternWithoutSplitCharViolenceCheck(pattern string)
// 注意这里返回的指针都是临时变量的内存地址
func (j *Json) checkPatternByPointer(key string, pointer *interface{}) *interface{} {
switch (*pointer).(type) {
case map[string]interface{}:
if v, ok := (*pointer).(map[string]interface{})[key]; ok {
return &v
}
case []interface{}:
if gstr.IsNumeric(key) {
n, err := strconv.Atoi(key)
if err == nil && len((*pointer).([]interface{})) > n {
return &(*pointer).([]interface{})[n]
}
case map[string]interface{}:
if v, ok := (*pointer).(map[string]interface{})[key]; ok {
return &v
}
case []interface{}:
if gstr.IsNumeric(key) {
n, err := strconv.Atoi(key)
if err == nil && len((*pointer).([]interface{})) > n {
return &(*pointer).([]interface{})[n]
}
}
}
return nil
}
@ -647,10 +655,10 @@ func (j *Json) ToMap() map[string]interface{} {
j.mu.RLock()
defer j.mu.RUnlock()
switch (*(j.p)).(type) {
case map[string]interface{}:
return (*(j.p)).(map[string]interface{})
default:
return nil
case map[string]interface{}:
return (*(j.p)).(map[string]interface{})
default:
return nil
}
}
@ -659,10 +667,10 @@ func (j *Json) ToArray() []interface{} {
j.mu.RLock()
defer j.mu.RUnlock()
switch (*(j.p)).(type) {
case []interface{}:
return (*(j.p)).([]interface{})
default:
return nil
case []interface{}:
return (*(j.p)).([]interface{})
default:
return nil
}
}

View File

@ -76,10 +76,12 @@ func View(name...string) *gview.View {
if path == "" {
path = genv.Get("GF_VIEWPATH")
if path == "" {
path = gfile.SelfDir()
if gfile.SelfDir() != gfile.TempDir() {
path = gfile.SelfDir()
}
}
}
view := gview.Get(path)
view := gview.New(path)
// 添加基于源码的搜索目录检索地址,常用于开发环境调试,只添加入口文件目录
if p := gfile.MainPkgPath(); p != "" && gfile.Exists(p) {
view.AddPath(p)
@ -103,7 +105,9 @@ func Config(file...string) *gcfg.Config {
if path == "" {
path = genv.Get("GF_CFGPATH")
if path == "" {
path = gfile.SelfDir()
if gfile.SelfDir() != gfile.TempDir() {
path = gfile.SelfDir()
}
}
}
config := gcfg.New(path, configFile)
@ -126,7 +130,8 @@ func Database(name...string) *gdb.Db {
db := instances.GetOrSetFuncLock(key, func() interface{} {
m := config.GetMap("database")
if m == nil {
glog.Errorfln(`incomplete configuration for database: "database" node not found in config file "%s"`, config.GetFilePath())
glog.Error(`database init failed: "database" node not found, is config file or configuration missing?`)
return nil
}
for group, v := range m {
cg := gdb.ConfigGroup{}

View File

@ -20,24 +20,23 @@ import (
// 请求对象
type Request struct {
http.Request
parsedGet bool // GET参数是否已经解析
parsedPost bool // POST参数是否已经解析
queryVars map[string][]string // GET参数
routerVars map[string][]string // 路由解析参数
exit bool // 是否退出当前请求流程执行
Id int // 请求id(唯一)
Server *Server // 请求关联的服务器对象
Cookie *Cookie // 与当前请求绑定的Cookie对象(并发安全)
Session *Session // 与当前请求绑定的Session对象(并发安全)
Response *Response // 对应请求的返回数据操作对象
Router *Router // 匹配到的路由对象
EnterTime int64 // 请求进入时间(微秒)
LeaveTime int64 // 请求完成时间(微秒)
Param interface{} // 开发者自定义参数
parsedHost string // 解析过后不带端口号的服务器域名名称
clientIp string // 解析过后的客户端IP地址
isFileRequest bool // 是否为静态文件请求(非服务请求,当静态文件存在时,优先级会被服务请求高,被识别为文件请求)
isFileServe bool // 是否为文件处理(调用Server.serveFile时设置为true), isFileRequest为true时isFileServe也为true
parsedGet bool // GET参数是否已经解析
parsedPost bool // POST参数是否已经解析
queryVars map[string][]string // GET参数
routerVars map[string][]string // 路由解析参数
exit bool // 是否退出当前请求流程执行
Id int // 请求id(唯一)
Server *Server // 请求关联的服务器对象
Cookie *Cookie // 与当前请求绑定的Cookie对象(并发安全)
Session *Session // 与当前请求绑定的Session对象(并发安全)
Response *Response // 对应请求的返回数据操作对象
Router *Router // 匹配到的路由对象
EnterTime int64 // 请求进入时间(微秒)
LeaveTime int64 // 请求完成时间(微秒)
params map[string]interface{} // 开发者自定义参数(请求流程中有效)
parsedHost string // 解析过后不带端口号的服务器域名名称
clientIp string // 解析过后的客户端IP地址
isFileRequest bool // 是否为静态文件请求(非服务请求,当静态文件存在时,优先级会被服务请求高,被识别为文件请求)
}
// 创建一个Request对象
@ -74,7 +73,8 @@ func (r *Request) Get(key string, def ... string) string {
return r.GetRequestString(key, def...)
}
func (r *Request) GetVar(key string, def ... interface{}) *gvar.Var {
// 建议都用该参数替代参数获取
func (r *Request) GetVar(key string, def ... interface{}) gvar.VarRead {
return r.GetRequestVar(key, def...)
}
@ -173,11 +173,6 @@ func (r *Request) IsFileRequest() bool {
return r.isFileRequest
}
// 判断请求是否为文件处理
func (r *Request) IsFileServe() bool {
return r.isFileServe
}
// 判断是否为AJAX请求
func (r *Request) IsAjaxRequest() bool {
return strings.EqualFold(r.Header.Get("X-Requested-With"), "XMLHttpRequest")
@ -203,7 +198,7 @@ func (r *Request) GetReferer() string {
return r.Header.Get("Referer")
}
// 获得结构体顶替的参数名称标签构成map返回
// 获得结构体对象的参数名称标签构成map返回
func (r *Request) getStructParamsTagMap(object interface{}) map[string]string {
tagmap := make(map[string]string)
fields := structs.Fields(object)

View File

@ -0,0 +1,28 @@
// Copyright 2017 gf Author(https://gitee.com/johng/gf). All Rights Reserved.
//
// This Source Code Form is subject to the terms of the MIT License.
// If a copy of the MIT was not distributed with this file,
// You can obtain one at https://gitee.com/johng/gf.
package ghttp
import "gitee.com/johng/gf/g/container/gvar"
// 设置请求流程共享变量
func (r *Request) SetParam(key string, value interface{}) {
if r.params == nil {
r.params = make(map[string]interface{})
}
r.params[key] = value
}
// 获取请求流程共享变量
func (r *Request) GetParam(key string) gvar.VarRead {
if r.params != nil {
if v, ok := r.params[key]; ok {
return gvar.New(v, false)
}
}
return gvar.New(nil, false)
}

View File

@ -26,15 +26,15 @@ func (r *Request) GetRequest(key string, def ... []string) []string {
return v
}
func (r *Request) GetRequestVar(key string, def ... interface{}) *gvar.Var {
func (r *Request) GetRequestVar(key string, def ... interface{}) gvar.VarRead {
value := r.GetRequest(key)
if value != nil {
return gvar.New(value)
return gvar.New(value, false)
}
if len(def) > 0 {
return gvar.New(def[0])
return gvar.New(def[0], false)
}
return nil
return gvar.New(nil, false)
}
func (r *Request) GetRequestString(key string, def ... string) string {

View File

@ -9,12 +9,12 @@ package ghttp
import (
"bytes"
"gitee.com/johng/gf/g/os/gfile"
"net/http"
"gitee.com/johng/gf/g/util/gconv"
"gitee.com/johng/gf/g/encoding/gparser"
"strconv"
"fmt"
"gitee.com/johng/gf/g/encoding/gparser"
"gitee.com/johng/gf/g/os/gfile"
"gitee.com/johng/gf/g/util/gconv"
"net/http"
"strconv"
)
// 服务端请求返回对象。
@ -158,11 +158,8 @@ func (r *Response) WriteStatus(status int, content...string) {
// 静态文件处理
func (r *Response) ServeFile(path string) {
r.request.isFileServe = true
// 首先判断是否给定的path已经是一个绝对路径
if !gfile.Exists(path) {
path, _ = r.Server.paths.Search(path)
}
path = gfile.RealPath(path)
if path == "" {
r.WriteStatus(http.StatusNotFound)
return
@ -170,10 +167,31 @@ func (r *Response) ServeFile(path string) {
r.Server.serveFile(r.request, path)
}
// 静态文件下载处理
func (r *Response) ServeFileDownload(path string, name...string) {
// 首先判断是否给定的path已经是一个绝对路径
path = gfile.RealPath(path)
if path == "" {
r.WriteStatus(http.StatusNotFound)
return
}
downloadName := ""
if len(name) > 0 {
downloadName = name[0]
} else {
downloadName = gfile.Basename(path)
}
r.Header().Set("Content-Type", "application/force-download")
r.Header().Set("Accept-Ranges", "bytes")
r.Header().Set("Content-Disposition", fmt.Sprintf(`attachment;filename="%s"`, downloadName))
r.Server.serveFile(r.request, path)
}
// 返回location标识引导客户端跳转
func (r *Response) RedirectTo(location string) {
r.Header().Set("Location", location)
r.WriteHeader(http.StatusFound)
r.request.Exit()
}
// 返回location标识引导客户端跳转到来源页面

View File

@ -15,10 +15,8 @@ import (
"gitee.com/johng/gf/g/container/gtype"
"gitee.com/johng/gf/g/os/gcache"
"gitee.com/johng/gf/g/os/genv"
"gitee.com/johng/gf/g/os/gfile"
"gitee.com/johng/gf/g/os/glog"
"gitee.com/johng/gf/g/os/gproc"
"gitee.com/johng/gf/g/os/gspath"
"gitee.com/johng/gf/g/os/gtime"
"gitee.com/johng/gf/g/util/gconv"
"gitee.com/johng/gf/g/util/gregex"
@ -33,6 +31,74 @@ import (
"time"
)
type (
// Server结构体
Server struct {
// 基本属性变量
name string // 服务名称,方便识别
config ServerConfig // 配置对象
servers []*gracefulServer // 底层http.Server列表
methodsMap map[string]struct{} // 所有支持的HTTP Method(初始化时自动填充)
servedCount *gtype.Int // 已经服务的请求数(4-8字节不考虑溢出情况)同时作为请求ID
// 服务注册相关
serveTree map[string]interface{} // 所有注册的服务回调函数(路由表,树型结构,哈希表+链表优先级匹配)
hooksTree map[string]interface{} // 所有注册的事件回调函数(路由表,树型结构,哈希表+链表优先级匹配)
serveCache *gcache.Cache // 服务注册路由内存缓存
hooksCache *gcache.Cache // 事件回调路由内存缓存
routesMap map[string][]registeredRouteItem // 已经注册的路由及对应的注册方法文件地址(用以路由重复注册判断)
// 自定义状态码回调
hsmu sync.RWMutex // status handler互斥锁
statusHandlerMap map[string]HandlerFunc // 不同状态码下的注册处理方法(例如404状态时的处理方法)
// SESSION
sessions *gcache.Cache // Session内存缓存
// Logger
logger *glog.Logger // 日志管理对象
}
// 路由对象
Router struct {
Uri string // 注册时的pattern - uri
Method string // 注册时的pattern - method
Domain string // 注册时的pattern - domain
RegRule string // 路由规则解析后对应的正则表达式
RegNames []string // 路由规则解析后对应的变量名称数组
Priority int // 优先级,用于链表排序,值越大优先级越高
}
// http回调函数注册信息
handlerItem struct {
name string // 注册的方法名称信息
rtype int // 注册方式(执行对象/回调函数/控制器)
ctype reflect.Type // 控制器类型(反射类型)
fname string // 回调方法名称
faddr HandlerFunc // 准确的执行方法内存地址(与以上两个参数二选一)
finit HandlerFunc // 初始化请求回调方法(执行对象注册方式下有效)
fshut HandlerFunc // 完成请求回调方法(执行对象注册方式下有效)
router *Router // 注册时绑定的路由对象
}
// 根据特定URL.Path解析后的路由检索结果项
handlerParsedItem struct {
handler *handlerItem // 路由注册项
values map[string][]string // 特定URL.Path的Router解析参数
}
// 已注册的路由项
registeredRouteItem struct {
file string // 文件路径及行数地址
handler *handlerItem // 路由注册项
}
// pattern与回调函数的绑定map
handlerMap map[string]*handlerItem
// HTTP注册函数
HandlerFunc func(r *Request)
// 文件描述符map
listenerFdMap map[string]string
)
const (
SERVER_STATUS_STOPPED = 0 // Server状态停止
SERVER_STATUS_RUNNING = 1 // Server状态运行
@ -42,8 +108,7 @@ const (
HOOK_AFTER_OUTPUT = "AfterOutput"
HOOK_BEFORE_CLOSE = "BeforeClose"
HOOK_AFTER_CLOSE = "AfterClose"
)
const (
gHTTP_METHODS = "GET,PUT,POST,DELETE,PATCH,HEAD,CONNECT,OPTIONS,TRACE"
gDEFAULT_SERVER = "default"
gDEFAULT_DOMAIN = "default"
@ -54,93 +119,27 @@ const (
gEXCEPTION_EXIT = "exit"
)
// ghttp.Server结构体
type Server struct {
// 基本属性变量
name string // 服务名称,方便识别
paths *gspath.SPath // 静态文件检索对象(类似nginx tryfile功能)
config ServerConfig // 配置对象
servers []*gracefulServer // 底层http.Server列表
methodsMap map[string]struct{} // 所有支持的HTTP Method(初始化时自动填充)
servedCount *gtype.Int // 已经服务的请求数(4-8字节不考虑溢出情况)同时作为请求ID
// 服务注册相关
serveTree map[string]interface{} // 所有注册的服务回调函数(路由表,树型结构,哈希表+链表优先级匹配)
hooksTree map[string]interface{} // 所有注册的事件回调函数(路由表,树型结构,哈希表+链表优先级匹配)
serveCache *gmap.StringInterfaceMap // 服务注册路由内存缓存
hooksCache *gmap.StringInterfaceMap // 事件回调路由内存缓存
routesMap map[string]registeredRouteItem // 已经注册的路由及对应的注册方法文件地址(用以路由重复注册判断)
// 自定义状态码回调
hsmu sync.RWMutex // status handler互斥锁
statusHandlerMap map[string]HandlerFunc // 不同状态码下的注册处理方法(例如404状态时的处理方法)
// SESSION
sessions *gcache.Cache // Session内存缓存
// Logger
logger *glog.Logger // 日志管理对象
}
var (
// Server表用以存储和检索名称与Server对象之间的关联关系
serverMapping = gmap.NewStringInterfaceMap()
// 路由对象
type Router struct {
Uri string // 注册时的pattern - uri
Method string // 注册时的pattern - method
Domain string // 注册时的pattern - domain
RegRule string // 路由规则解析后对应的正则表达式
RegNames []string // 路由规则解析后对应的变量名称数组
Priority int // 优先级,用于链表排序,值越大优先级越高
}
// 正常运行的Server数量如果没有运行、失败或者全部退出那么该值为0
serverRunning = gtype.NewInt()
// pattern与回调函数的绑定map
type handlerMap map[string]*handlerItem
// Web Socket默认配置
wsUpgrader = websocket.Upgrader {
// 默认允许WebSocket请求跨域权限控制可以由业务层自己负责灵活度更高
CheckOrigin: func(r *http.Request) bool {
return true
},
}
// Web Server已完成服务事件通道当有事件时表示服务完成当前进程退出
doneChan = make(chan struct{}, 1000)
// http回调函数注册信息
type handlerItem struct {
name string // 注册的方法名称信息
rtype int // 注册方式(执行对象/回调函数/控制器)
ctype reflect.Type // 控制器类型(反射类型)
fname string // 回调方法名称
faddr HandlerFunc // 准确的执行方法内存地址(与以上两个参数二选一)
finit HandlerFunc // 初始化请求回调方法(执行对象注册方式下有效)
fshut HandlerFunc // 完成请求回调方法(执行对象注册方式下有效)
router *Router // 注册时绑定的路由对象
}
// 用于服务进程初始化,只能初始化一次,采用“懒初始化”(在server运行时才初始化)
serverProcInited = gtype.NewBool()
)
// 根据特定URL.Path解析后的路由检索结果项
type handlerParsedItem struct {
handler *handlerItem // 路由注册项
values map[string][]string // 特定URL.Path的Router解析参数
}
// 已注册的路由项
type registeredRouteItem struct {
file string // 文件路径及行数地址
handler *handlerItem // 路由注册项
}
// HTTP注册函数
type HandlerFunc func(r *Request)
// 文件描述符map
type listenerFdMap map[string]string
// Server表用以存储和检索名称与Server对象之间的关联关系
var serverMapping = gmap.NewStringInterfaceMap()
// 正常运行的Server数量如果没有运行、失败或者全部退出那么该值为0
var serverRunning = gtype.NewInt()
// Web Socket默认配置
var wsUpgrader = websocket.Upgrader {
// 默认允许WebSocket请求跨域权限控制可以由业务层自己负责灵活度更高
CheckOrigin: func(r *http.Request) bool {
return true
},
}
// Web Server已完成服务事件通道当有事件时表示服务完成当前进程退出
var doneChan = make(chan struct{}, 1000)
// 用于服务进程初始化,只能初始化一次,采用“懒初始化”(在server运行时才初始化)
var serverProcInited = gtype.NewBool()
// Web Server进程初始化.
// 注意该方法不能放置于包初始化方法init中不使用ghttp.Server的功能便不能初始化对应的协程goroutine逻辑.
@ -177,15 +176,14 @@ func GetServer(name...interface{}) (*Server) {
}
s := &Server {
name : sname,
paths : gspath.New(),
servers : make([]*gracefulServer, 0),
methodsMap : make(map[string]struct{}),
statusHandlerMap : make(map[string]HandlerFunc),
serveTree : make(map[string]interface{}),
hooksTree : make(map[string]interface{}),
serveCache : gmap.NewStringInterfaceMap(),
hooksCache : gmap.NewStringInterfaceMap(),
routesMap : make(map[string]registeredRouteItem),
serveCache : gcache.New(),
hooksCache : gcache.New(),
routesMap : make(map[string][]registeredRouteItem),
sessions : gcache.New(),
servedCount : gtype.NewInt(),
logger : glog.New(),
@ -213,30 +211,11 @@ func (s *Server) Start() error {
return errors.New("server is already running")
}
// 如果设置了静态文件目录,那么优先按照静态文件目录进行检索,其次是当前可执行文件工作目录;
// 并且如果是开发环境默认也会添加main包的源码目录路径做为二级检索。
if s.config.ServerRoot != "" {
if rp, err := s.paths.Set(s.config.ServerRoot); err != nil {
glog.Error("ghttp.SetServerRoot failed:", err.Error())
return err
} else {
glog.Debug("ghttp.SetServerRoot:", rp)
}
}
// 添加当前可执行文件运行目录到搜索目录
s.paths.Add(gfile.SelfDir())
// (开发环境)添加main源码包到搜索目录
if p := gfile.MainPkgPath(); p != "" && gfile.Exists(p) {
s.paths.Add(p)
}
// (安全控制)不能访问当前执行文件
s.paths.Remove(gfile.SelfPath())
// 底层http server配置
if s.config.Handler == nil {
s.config.Handler = http.HandlerFunc(s.defaultHttpHandle)
}
// 不允许访问的路由注册(通过HOOK实现)
// 不允许访问的路由注册(使用HOOK实现)
if s.config.DenyRoutes != nil {
for _, v := range s.config.DenyRoutes {
s.BindHookHandler(v, HOOK_BEFORE_SERVE, func(r *Request) {
@ -246,18 +225,6 @@ func (s *Server) Start() error {
}
}
// 配置相关相对路径处理
if s.config.HTTPSCertPath != "" && !gfile.Exists(s.config.HTTPSCertPath) {
if t, _ := s.paths.Search(s.config.HTTPSCertPath); t != "" {
s.config.HTTPSCertPath = t
}
}
if s.config.HTTPSKeyPath != "" && !gfile.Exists(s.config.HTTPSKeyPath) {
if t, _ := s.paths.Search(s.config.HTTPSKeyPath); t != "" {
s.config.HTTPSKeyPath = t
}
}
// gzip压缩文件类型
//if s.config.GzipContentTypes != nil {
// for _, v := range s.config.GzipContentTypes {
@ -282,7 +249,9 @@ func (s *Server) Start() error {
// 如果是子进程,那么服务开启后通知父进程销毁
if gproc.IsChild() {
gtime.SetTimeout(2*time.Second, func() {
gproc.Send(gproc.PPid(), []byte("exit"), gADMIN_GPROC_COMM_GROUP)
if err := gproc.Send(gproc.PPid(), []byte("exit"), gADMIN_GPROC_COMM_GROUP); err != nil {
panic(err)
}
})
}
@ -304,11 +273,12 @@ func (s *Server) DumpRoutesMap() {
// 获得路由表(格式化字符串)
func (s *Server) GetRouteMap() string {
type tableItem struct {
hook string
domain string
method string
route string
handler string
hook string
domain string
method string
route string
handler string
priority int
}
buf := bytes.NewBuffer(nil)
@ -319,31 +289,37 @@ func (s *Server) GetRouteMap() string {
table.SetCenterSeparator("|")
m := make(map[string]*garray.SortedArray)
for k, v := range s.routesMap {
for k, registeredItems := range s.routesMap {
array, _ := gregex.MatchString(`(.*?)%([A-Z]+):(.+)@(.+)`, k)
item := &tableItem{
hook : array[1],
domain : array[4],
method : array[2],
route : array[3],
handler : v.handler.name,
}
if _, ok := m[item.domain]; !ok {
m[item.domain] = garray.NewSortedArray(100, func(v1, v2 interface{}) int {
item1 := v1.(*tableItem)
item2 := v2.(*tableItem)
r := 0
if r = strings.Compare(item1.domain, item2.domain); r == 0 {
if r = strings.Compare(item1.route, item2.route); r == 0 {
if r = strings.Compare(item1.method, item2.method); r == 0 {
r = strings.Compare(item1.hook, item2.hook)
for index, registeredItem := range registeredItems {
item := &tableItem {
hook : array[1],
domain : array[4],
method : array[2],
route : array[3],
handler : registeredItem.handler.name,
priority : len(registeredItems) - index - 1,
}
if _, ok := m[item.domain]; !ok {
// 注意排序函数的逻辑
m[item.domain] = garray.NewSortedArray(100, func(v1, v2 interface{}) int {
item1 := v1.(*tableItem)
item2 := v2.(*tableItem)
r := 0
if r = strings.Compare(item1.domain, item2.domain); r == 0 {
if r = strings.Compare(item1.route, item2.route); r == 0 {
if r = strings.Compare(item1.method, item2.method); r == 0 {
if r = strings.Compare(item1.hook, item2.hook); r == 0 {
r = item2.priority - item1.priority
}
}
}
}
}
return r
}, false)
return r
}, false)
}
m[item.domain].Add(item)
}
m[item.domain].Add(item)
}
addr := s.config.Addr
if s.config.HTTPSAddr != "" {
@ -357,7 +333,7 @@ func (s *Server) GetRouteMap() string {
data[1] = addr
data[2] = item.domain
data[3] = item.method
data[4] = gconv.String(len(strings.Split(item.route, "/")) - 1)
data[4] = gconv.String(len(strings.Split(item.route, "/")) - 1 + item.priority)
data[5] = item.route
data[6] = item.handler
data[7] = item.hook
@ -482,7 +458,7 @@ func (s *Server) startServer(fdMap listenerFdMap) {
serverRunning.Add(-1)
// 如果非关闭错误,那么提示报错,否则认为是正常的服务关闭操作
if err != nil && !strings.EqualFold(http.ErrServerClosed.Error(), err.Error()) {
glog.Error(err)
glog.Fatal(err)
}
// 如果所有异步的Server都已经停止并且没有在管理操作(重启/关闭)进行中那么主Server就可以退出了
if serverRunning.Val() < 1 && serverProcessStatus.Val() == 0 {

View File

@ -50,16 +50,18 @@ var serverProcessStatus = gtype.NewInt()
// 服务管理首页
func (p *utilAdmin) Index(r *Request) {
data := map[string]interface{}{
"pid" : gproc.Pid(),
"uri" : strings.TrimRight(r.URL.Path, "/"),
}
buffer, _ := gview.ParseContent(`
<html>
<head>
<title>gf ghttp admin</title>
<title>GoFrame Web Server Admin</title>
</head>
<body>
<p><a href="{{$.uri}}/restart">restart</a></p>
<p><a href="{{$.uri}}/shutdown">shutdown</a></p>
<p>PID: {{.pid}}</p>
<p><a href="{{$.uri}}/restart">Restart</a></p>
<p><a href="{{$.uri}}/shutdown">Shutdown</a></p>
</body>
</html>
`, data)

View File

@ -25,7 +25,6 @@ func handleProcessSignal() {
syscall.SIGINT,
syscall.SIGQUIT,
syscall.SIGKILL,
syscall.SIGHUP,
syscall.SIGTERM,
syscall.SIGUSR1,
syscall.SIGUSR2,
@ -34,7 +33,7 @@ func handleProcessSignal() {
sig = <- procSignalChan
switch sig {
// 进程终止,停止所有子进程运行
case syscall.SIGINT, syscall.SIGQUIT, syscall.SIGKILL, syscall.SIGHUP, syscall.SIGTERM:
case syscall.SIGINT, syscall.SIGQUIT, syscall.SIGKILL, syscall.SIGTERM:
shutdownWebServers(sig.String())
return

View File

@ -3,17 +3,16 @@
// This Source Code Form is subject to the terms of the MIT License.
// If a copy of the MIT was not distributed with this file,
// You can obtain one at https://gitee.com/johng/gf.
// 配置管理数据结构定义.
package ghttp
import (
"time"
"fmt"
"gitee.com/johng/gf/g/os/gfile"
"gitee.com/johng/gf/g/os/glog"
"net/http"
"strconv"
"strings"
"gitee.com/johng/gf/g/os/glog"
"gitee.com/johng/gf/g/os/gfile"
"time"
)
const (
@ -36,75 +35,87 @@ type LogHandler func(r *Request, error ... interface{})
// HTTP Server 设置结构体,静态配置
type ServerConfig struct {
// 底层http对象配置
Addr string // 监听IP和端口监听本地所有IP使用":端口"(支持多个地址,使用","号分隔)
HTTPSAddr string // HTTPS服务监听地址(支持多个地址,使用","号分隔)
HTTPSCertPath string // HTTPS证书文件路径
HTTPSKeyPath string // HTTPS签名文件路径
Handler http.Handler // 默认的处理函数
ReadTimeout time.Duration // 读取超时
WriteTimeout time.Duration // 写入超时
IdleTimeout time.Duration // 等待超时
MaxHeaderBytes int // 最大的header长度
Addr string // 监听IP和端口监听本地所有IP使用":端口"(支持多个地址,使用","号分隔)
HTTPSAddr string // HTTPS服务监听地址(支持多个地址,使用","号分隔)
HTTPSCertPath string // HTTPS证书文件路径
HTTPSKeyPath string // HTTPS签名文件路径
Handler http.Handler // 默认的处理函数
ReadTimeout time.Duration // 读取超时
WriteTimeout time.Duration // 写入超时
IdleTimeout time.Duration // 等待超时
MaxHeaderBytes int // 最大的header长度
// 静态文件配置
IndexFiles []string // 默认访问的文件列表
IndexFolder bool // 如果访问目录是否显示目录列表
ServerAgent string // server agent
ServerRoot string // 服务器服务的本地目录根路径
IndexFiles []string // 默认访问的文件列表
IndexFolder bool // 如果访问目录是否显示目录列表
ServerAgent string // Server Agent
ServerRoot string // 服务器服务的本地目录根路径(检索优先级比StaticPaths低)
SearchPaths []string // 静态文件搜索目录(包含ServerRoot按照优先级进行排序)
StaticPaths []staticPathItem // 静态文件目录映射(按照优先级进行排序)
FileServerEnabled bool // 是否允许静态文件服务(总开关,默认开启)
// COOKIE
CookieMaxAge int // Cookie有效期
CookiePath string // Cookie有效Path(注意同时也会影响SessionID)
CookieDomain string // Cookie有效Domain(注意同时也会影响SessionID)
CookieMaxAge int // Cookie有效期
CookiePath string // Cookie有效Path(注意同时也会影响SessionID)
CookieDomain string // Cookie有效Domain(注意同时也会影响SessionID)
// SESSION
SessionMaxAge int // Session有效期
SessionIdName string // SessionId名称
SessionMaxAge int // Session有效期
SessionIdName string // SessionId名称
// IP访问控制
DenyIps []string // 不允许访问的ip列表支持ip前缀过滤如: 10 将不允许10开头的ip访问
AllowIps []string // 仅允许访问的ip列表支持ip前缀过滤如: 10 将仅允许10开头的ip访问
// ip访问控制
DenyIps []string // 不允许访问的ip列表支持ip前缀过滤如: 10 将不允许10开头的ip访问
AllowIps []string // 仅允许访问的ip列表支持ip前缀过滤如: 10 将仅允许10开头的ip访问
// 路由访问控制
DenyRoutes []string // 不允许访问的路由规则列表
DenyRoutes []string // 不允许访问的路由规则列表
Rewrites map[string]string // URI Rewrite重写配置
// 日志配置
LogPath string // 存放日志的目录路径
LogHandler LogHandler // 自定义日志处理回调方法
ErrorLogEnabled bool // 是否开启error log
AccessLogEnabled bool // 是否开启access log
LogPath string // 存放日志的目录路径
LogHandler LogHandler // 自定义日志处理回调方法
ErrorLogEnabled bool // 是否开启error log
AccessLogEnabled bool // 是否开启access log
// 其他设置
NameToUriType int // 服务注册时对象和方法名称转换为URI时的规则
GzipContentTypes []string // 允许进行gzip压缩的文件类型
DumpRouteMap bool // 是否在程序启动时默认打印路由表信息
NameToUriType int // 服务注册时对象和方法名称转换为URI时的规则
GzipContentTypes []string // 允许进行gzip压缩的文件类型
DumpRouteMap bool // 是否在程序启动时默认打印路由表信息
RouterCacheExpire int // 路由检索缓存过期时间(秒)
}
// 默认HTTP Server
// 默认HTTP Server配置
var defaultServerConfig = ServerConfig {
Addr : "",
HTTPSAddr : "",
Handler : nil,
ReadTimeout : 60 * time.Second,
WriteTimeout : 60 * time.Second,
IdleTimeout : 60 * time.Second,
MaxHeaderBytes : 1024,
IndexFiles : []string{"index.html", "index.htm"},
IndexFolder : false,
ServerAgent : "gf",
ServerRoot : "",
Addr : "",
HTTPSAddr : "",
Handler : nil,
ReadTimeout : 60 * time.Second,
WriteTimeout : 60 * time.Second,
IdleTimeout : 60 * time.Second,
MaxHeaderBytes : 1024,
CookieMaxAge : gDEFAULT_COOKIE_MAX_AGE,
CookiePath : gDEFAULT_COOKIE_PATH,
CookieDomain : "",
IndexFiles : []string{"index.html", "index.htm"},
IndexFolder : false,
ServerAgent : "gf",
ServerRoot : "",
StaticPaths : make([]staticPathItem, 0),
FileServerEnabled : true,
SessionMaxAge : gDEFAULT_SESSION_MAX_AGE,
SessionIdName : gDEFAULT_SESSION_ID_NAME,
CookieMaxAge : gDEFAULT_COOKIE_MAX_AGE,
CookiePath : gDEFAULT_COOKIE_PATH,
CookieDomain : "",
ErrorLogEnabled : true,
SessionMaxAge : gDEFAULT_SESSION_MAX_AGE,
SessionIdName : gDEFAULT_SESSION_ID_NAME,
GzipContentTypes : defaultGzipContentTypes,
ErrorLogEnabled : true,
DumpRouteMap : true,
GzipContentTypes : defaultGzipContentTypes,
DumpRouteMap : true,
RouterCacheExpire : 60,
Rewrites : make(map[string]string),
}
// 获取默认的http server设置
@ -117,6 +128,7 @@ func Config() ServerConfig {
func (s *Server)SetConfig(c ServerConfig) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
return
}
if c.Handler == nil {
c.Handler = http.HandlerFunc(s.defaultHttpHandle)
@ -132,6 +144,7 @@ func (s *Server)SetConfig(c ServerConfig) {
func (s *Server)SetAddr(addr string) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
return
}
s.config.Addr = addr
}
@ -156,6 +169,7 @@ func (s *Server)SetPort(port...int) {
func (s *Server)SetHTTPSAddr(addr string) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
return
}
s.config.HTTPSAddr = addr
}
@ -164,6 +178,7 @@ func (s *Server)SetHTTPSAddr(addr string) {
func (s *Server)SetHTTPSPort(port...int) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
return
}
if len(port) > 0 {
s.config.HTTPSAddr = ""
@ -180,15 +195,31 @@ func (s *Server)SetHTTPSPort(port...int) {
func (s *Server)EnableHTTPS(certFile, keyFile string) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
return
}
s.config.HTTPSCertPath = certFile
s.config.HTTPSKeyPath = keyFile
certFileRealPath := gfile.RealPath(certFile)
if certFileRealPath == "" {
certFileRealPath = gfile.RealPath(gfile.MainPkgPath() + gfile.Separator + certFileRealPath)
}
if certFileRealPath == "" {
glog.Fatal(fmt.Sprintf(`[ghttp] EnableHTTPS failed: certFile "%s" does not exist`, certFile))
}
keyFileRealPath := gfile.RealPath(keyFile)
if keyFileRealPath == "" {
keyFileRealPath = gfile.RealPath(gfile.MainPkgPath() + gfile.Separator + keyFileRealPath)
}
if keyFileRealPath == "" {
glog.Fatal(fmt.Sprintf(`[ghttp] EnableHTTPS failed: keyFile "%s" does not exist`, keyFile))
}
s.config.HTTPSCertPath = certFileRealPath
s.config.HTTPSKeyPath = keyFileRealPath
}
// 设置http server参数 - ReadTimeout
func (s *Server)SetReadTimeout(t time.Duration) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
return
}
s.config.ReadTimeout = t
}
@ -197,6 +228,7 @@ func (s *Server)SetReadTimeout(t time.Duration) {
func (s *Server)SetWriteTimeout(t time.Duration) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
return
}
s.config.WriteTimeout = t
}
@ -205,6 +237,7 @@ func (s *Server)SetWriteTimeout(t time.Duration) {
func (s *Server)SetIdleTimeout(t time.Duration) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
return
}
s.config.IdleTimeout = t
}
@ -213,74 +246,25 @@ func (s *Server)SetIdleTimeout(t time.Duration) {
func (s *Server)SetMaxHeaderBytes(b int) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
return
}
s.config.MaxHeaderBytes = b
}
// 设置http server参数 - IndexFiles默认展示文件index.html, index.htm
func (s *Server)SetIndexFiles(index []string) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
}
s.config.IndexFiles = index
}
// 允许展示访问目录的文件列表
func (s *Server)SetIndexFolder(index bool) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
}
s.config.IndexFolder = index
}
// 设置http server参数 - ServerAgent
func (s *Server)SetServerAgent(agent string) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
return
}
s.config.ServerAgent = agent
}
// 设置http server参数 - ServerRoot
func (s *Server)SetServerRoot(root string) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
}
// RealPath的作用除了校验地址正确性以外还转换分隔符号为当前系统正确的文件分隔符号
path := gfile.RealPath(root)
if path == "" {
glog.Error("invalid root path \"" + root + "\"")
}
s.config.ServerRoot = strings.TrimRight(path, gfile.Separator)
}
func (s *Server) SetDenyIps(ips []string) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
}
s.config.DenyIps = ips
}
func (s *Server) SetAllowIps(ips []string) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
}
s.config.AllowIps = ips
}
func (s *Server) SetDenyRoutes(routes []string) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
}
s.config.DenyRoutes = routes
}
func (s *Server) SetGzipContentTypes(types []string) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
return
}
s.config.GzipContentTypes = types
}
@ -289,6 +273,7 @@ func (s *Server) SetGzipContentTypes(types []string) {
func (s *Server) SetNameToUriType(t int) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
return
}
s.config.NameToUriType = t
}
@ -297,22 +282,21 @@ func (s *Server) SetNameToUriType(t int) {
func (s *Server) SetDumpRouteMap(enabled bool) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
return
}
s.config.DumpRouteMap = enabled
}
// 添加静态文件搜索目录,必须给定目录的绝对路径
func (s *Server) AddSearchPath(path string) error {
if rp, err := s.paths.Add(path); err != nil {
glog.Error("ghttp.AddSearchPath failed:", err.Error())
return err
} else {
glog.Debug("ghttp.AddSearchPath:", rp)
// 设置路由缓存过期时间(秒)
func (s *Server) SetRouterCacheExpire(expire int) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
return
}
return nil
s.config.RouterCacheExpire = expire
}
// 获取
// 获取WebServer名称
func (s *Server) GetName() string {
return s.name
}

View File

@ -3,7 +3,6 @@
// This Source Code Form is subject to the terms of the MIT License.
// If a copy of the MIT was not distributed with this file,
// You can obtain one at https://gitee.com/johng/gf.
// 配置管理数据结构定义.
package ghttp
@ -16,6 +15,7 @@ import (
func (s *Server)SetCookieMaxAge(age int) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
return
}
s.config.CookieMaxAge = age
}
@ -24,6 +24,7 @@ func (s *Server)SetCookieMaxAge(age int) {
func (s *Server)SetCookiePath(path string) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
return
}
s.config.CookiePath = path
}
@ -32,6 +33,7 @@ func (s *Server)SetCookiePath(path string) {
func (s *Server)SetCookieDomain(domain string) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
return
}
s.config.CookieDomain = domain
}

View File

@ -3,7 +3,6 @@
// This Source Code Form is subject to the terms of the MIT License.
// If a copy of the MIT was not distributed with this file,
// You can obtain one at https://gitee.com/johng/gf.
// 配置管理数据结构定义.
package ghttp
@ -15,6 +14,7 @@ import (
func (s *Server)SetLogPath(path string) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
return
}
if len(path) == 0 {
return
@ -27,6 +27,7 @@ func (s *Server)SetLogPath(path string) {
func (s *Server)SetAccessLogEnabled(enabled bool) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
return
}
s.config.AccessLogEnabled = enabled
}
@ -35,6 +36,7 @@ func (s *Server)SetAccessLogEnabled(enabled bool) {
func (s *Server)SetErrorLogEnabled(enabled bool) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
return
}
s.config.ErrorLogEnabled = enabled
}
@ -43,6 +45,7 @@ func (s *Server)SetErrorLogEnabled(enabled bool) {
func (s *Server) SetLogHandler(handler LogHandler) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
return
}
s.config.LogHandler = handler
}

View File

@ -0,0 +1,54 @@
// Copyright 2018 gf Author(https://gitee.com/johng/gf). All Rights Reserved.
//
// This Source Code Form is subject to the terms of the MIT License.
// If a copy of the MIT was not distributed with this file,
// You can obtain one at https://gitee.com/johng/gf.
package ghttp
import "gitee.com/johng/gf/g/os/glog"
func (s *Server) SetDenyIps(ips []string) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
return
}
s.config.DenyIps = ips
}
func (s *Server) SetAllowIps(ips []string) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
return
}
s.config.AllowIps = ips
}
func (s *Server) SetDenyRoutes(routes []string) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
return
}
s.config.DenyRoutes = routes
}
// 设置URI重写规则
func (s *Server) SetRewrite(uri string, rewrite string) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
return
}
s.config.Rewrites[uri] = rewrite
}
// 设置URI重写规则批量
func (s *Server) SetRewriteMap(rewrites map[string]string) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
return
}
for k, v := range rewrites {
s.config.Rewrites[k] = v
}
}

View File

@ -3,7 +3,6 @@
// This Source Code Form is subject to the terms of the MIT License.
// If a copy of the MIT was not distributed with this file,
// You can obtain one at https://gitee.com/johng/gf.
// 配置管理数据结构定义.
package ghttp
@ -13,6 +12,7 @@ import "gitee.com/johng/gf/g/os/glog"
func (s *Server) SetSessionMaxAge(age int) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
return
}
s.config.SessionMaxAge = age
}
@ -21,6 +21,7 @@ func (s *Server) SetSessionMaxAge(age int) {
func (s *Server) SetSessionIdName(name string) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
return
}
s.config.SessionIdName = name
}

View File

@ -0,0 +1,136 @@
// Copyright 2017 gf Author(https://gitee.com/johng/gf). All Rights Reserved.
//
// This Source Code Form is subject to the terms of the MIT License.
// If a copy of the MIT was not distributed with this file,
// You can obtain one at https://gitee.com/johng/gf.
// 静态文件搜索优先级: ServerPaths > ServerRoot > SearchPath
package ghttp
import (
"fmt"
"gitee.com/johng/gf/g/container/garray"
"gitee.com/johng/gf/g/os/gfile"
"gitee.com/johng/gf/g/os/glog"
"gitee.com/johng/gf/g/util/gconv"
"strings"
)
// 静态文件目录映射关系对象
type staticPathItem struct {
prefix string // 映射的URI前缀
path string // 静态文件目录绝对路径
}
// 设置http server参数 - IndexFiles默认展示文件index.html, index.htm
func (s *Server)SetIndexFiles(index []string) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
return
}
s.config.IndexFiles = index
}
// 允许展示访问目录的文件列表
func (s *Server)SetIndexFolder(enabled bool) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
return
}
s.config.IndexFolder = enabled
}
// 是否开启/关闭静态文件服务,当关闭时仅提供动态接口服务,路由性能会得到一定提升
func (s *Server) SetFileServerEnabled(enabled bool) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
return
}
s.config.FileServerEnabled = enabled
}
// 设置http server参数 - ServerRoot
func (s *Server)SetServerRoot(root string) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
return
}
// RealPath的作用除了校验地址正确性以外还转换分隔符号为当前系统正确的文件分隔符号
path := gfile.RealPath(root)
if path == "" {
path = gfile.RealPath(gfile.MainPkgPath() + gfile.Separator + root)
}
if path == "" {
glog.Fatal(fmt.Sprintf(`[ghttp] SetServerRoot failed: path "%s" does not exist`, root))
}
s.config.SearchPaths = []string{strings.TrimRight(path, gfile.Separator)}
}
// 添加静态文件搜索**目录**,必须给定目录的绝对路径
func (s *Server) AddSearchPath(path string) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
return
}
// RealPath的作用除了校验地址正确性以外还转换分隔符号为当前系统正确的文件分隔符号
realPath := gfile.RealPath(path)
if realPath == "" {
realPath = gfile.RealPath(gfile.MainPkgPath() + gfile.Separator + path)
}
if realPath == "" {
glog.Fatal(fmt.Sprintf(`[ghttp] AddSearchPath failed: path "%s" does not exist`, path))
}
s.config.SearchPaths = append(s.config.SearchPaths, realPath)
}
// 添加URI与静态**目录**的映射
func (s *Server) AddStaticPath(prefix string, path string) {
if s.Status() == SERVER_STATUS_RUNNING {
glog.Error(gCHANGE_CONFIG_WHILE_RUNNING_ERROR)
return
}
// RealPath的作用除了校验地址正确性以外还转换分隔符号为当前系统正确的文件分隔符号
realPath := gfile.RealPath(path)
if realPath == "" {
realPath = gfile.RealPath(gfile.MainPkgPath() + gfile.Separator + path)
}
if realPath == "" {
glog.Fatal(fmt.Sprintf(`[ghttp] AddStaticPath failed: path "%s" does not exist`, path))
}
addItem := staticPathItem {
prefix : prefix,
path : realPath,
}
if len(s.config.StaticPaths) > 0 {
// 先添加item
s.config.StaticPaths = append(s.config.StaticPaths, addItem)
// 按照prefix从长到短进行排序
array := garray.NewSortedArray(0, func(v1, v2 interface{}) int {
s1 := gconv.String(v1)
s2 := gconv.String(v2)
r := len(s2) - len(s1)
if r == 0 {
r = strings.Compare(s1, s2)
}
return r
}, false)
for _, v := range s.config.StaticPaths {
array.Add(v.prefix)
}
// 按照重新排序的顺序重新添加item
paths := make([]staticPathItem, 0)
for _, v := range array.Slice() {
for _, item := range s.config.StaticPaths {
if strings.EqualFold(gconv.String(v), item.prefix) {
paths = append(paths, item)
break
}
}
}
s.config.StaticPaths = paths
} else {
s.config.StaticPaths = []staticPathItem { addItem }
}
}

View File

@ -35,39 +35,40 @@ type CookieItem struct {
httpOnly bool
}
// 获取或者创建一个cookie对象,与传入的请求对应
// 获取或者创建一个COOKIE对象,与传入的请求对应(延迟初始化)
func GetCookie(r *Request) *Cookie {
if r.Cookie != nil {
return r.Cookie
}
r.Cookie = &Cookie {
data : make(map[string]CookieItem),
path : r.Server.GetCookiePath(),
domain : r.Server.GetCookieDomain(),
maxage : r.Server.GetCookieMaxAge(),
server : r.Server,
request : r,
response : r.Response,
return &Cookie {
request : r,
}
// 默认有效域名
if r.Cookie.domain == "" {
r.Cookie.domain = r.GetHost()
}
r.Cookie.init()
return r.Cookie
}
// 从请求流中初始化,无锁
// 从请求流中初始化,无锁,延迟初始化
func (c *Cookie) init() {
for _, v := range c.request.Cookies() {
c.data[v.Name] = CookieItem {
v.Value, v.Domain, v.Path, v.Expires.Second(), v.HttpOnly,
if c.data == nil {
c.data = make(map[string]CookieItem)
c.path = c.request.Server.GetCookiePath()
c.domain = c.request.Server.GetCookieDomain()
c.maxage = c.request.Server.GetCookieMaxAge()
c.server = c.request.Server
c.response = c.request.Response
// 如果没有设置COOKIE有效域名那么设置HOST为默认有效域名
if c.domain == "" {
c.domain = c.request.GetHost()
}
for _, v := range c.request.Cookies() {
c.data[v.Name] = CookieItem {
v.Value, v.Domain, v.Path, v.Expires.Second(), v.HttpOnly,
}
}
}
}
// 获取所有的Cookie并构造成map返回
// 获取所有的Cookie并构造成map[string]string返回.
func (c *Cookie) Map() map[string]string {
c.init()
m := make(map[string]string)
for k, v := range c.data {
m[k] = v.value
@ -77,6 +78,7 @@ func (c *Cookie) Map() map[string]string {
// 获取SessionId不存在时则创建
func (c *Cookie) SessionId() string {
c.init()
id := c.Get(c.server.GetSessionIdName())
if id == "" {
id = makeSessionId()
@ -87,6 +89,7 @@ func (c *Cookie) SessionId() string {
// 判断Cookie中是否存在制定键名(并且没有过期)
func (c *Cookie) Contains(key string) bool {
c.init()
if r, ok := c.data[key]; ok {
if r.expire >= 0 {
return true
@ -95,11 +98,6 @@ func (c *Cookie) Contains(key string) bool {
return false
}
// 设置SessionId
func (c *Cookie) SetSessionId(id string) {
c.Set(c.server.GetSessionIdName(), id)
}
// 设置cookie使用默认参数
func (c *Cookie) Set(key, value string) {
c.SetCookie(key, value, c.domain, c.path, c.server.GetCookieMaxAge())
@ -107,6 +105,7 @@ func (c *Cookie) Set(key, value string) {
// 设置cookie带详细cookie参数
func (c *Cookie) SetCookie(key, value, domain, path string, maxAge int, httpOnly ... bool) {
c.init()
isHttpOnly := false
if len(httpOnly) > 0 {
isHttpOnly = httpOnly[0]
@ -116,8 +115,14 @@ func (c *Cookie) SetCookie(key, value, domain, path string, maxAge int, httpOnly
}
}
// 设置SessionId
func (c *Cookie) SetSessionId(id string) {
c.Set(c.server.GetSessionIdName(), id)
}
// 查询cookie
func (c *Cookie) Get(key string) string {
c.init()
if r, ok := c.data[key]; ok {
if r.expire >= 0 {
return r.value

View File

@ -10,6 +10,7 @@ package ghttp
import (
"fmt"
"gitee.com/johng/gf/g/encoding/ghtml"
"gitee.com/johng/gf/g/os/gspath"
"gitee.com/johng/gf/g/os/gtime"
"net/http"
"os"
@ -28,9 +29,17 @@ func (s *Server)defaultHttpHandle(w http.ResponseWriter, r *http.Request) {
// 其次,如果没有对应的自定义处理接口配置,那么走默认的域名处理接口配置;
// 最后,如果以上都没有找到处理接口,那么进行文件处理;
func (s *Server)handleRequest(w http.ResponseWriter, r *http.Request) {
// 重写规则判断
if len(s.config.Rewrites) > 0 {
if rewrite, ok := s.config.Rewrites[r.URL.Path]; ok {
r.URL.Path = rewrite
}
}
// 去掉末尾的"/"号
if r.URL.Path != "/" {
r.URL.Path = strings.TrimRight(r.URL.Path, "/")
for r.URL.Path[len(r.URL.Path) - 1] == '/' {
r.URL.Path = r.URL.Path[:len(r.URL.Path) - 1]
}
}
// 创建请求处理对象
@ -57,10 +66,14 @@ func (s *Server)handleRequest(w http.ResponseWriter, r *http.Request) {
// 静态文件 > 动态服务 > 静态目录
// ============================================================
staticFile := ""
isStaticDir := false
// 优先执行静态文件检索(检测是否存在对应的静态文件包括index files处理)
staticFile, isStaticDir := s.paths.Search(r.URL.Path, s.config.IndexFiles...)
if staticFile != "" {
request.isFileRequest = true
if s.config.FileServerEnabled {
staticFile, isStaticDir = s.searchStaticFile(r.URL.Path)
if staticFile != "" {
request.isFileRequest = true
}
}
// 动态服务检索
@ -75,13 +88,18 @@ func (s *Server)handleRequest(w http.ResponseWriter, r *http.Request) {
}
}
// 判断最终对该请求提供的服务方式
if isStaticDir && handler != nil {
request.isFileRequest = false
}
// 事件 - BeforeServe
s.callHookHandler(HOOK_BEFORE_SERVE, request)
// 执行静态文件服务/回调控制器/执行对象/方法
if !request.IsExited() {
// 需要再次判断文件是否真实存在,因为文件检索可能使用了缓存,从健壮性考虑这里需要二次判断
if request.IsFileRequest() && !isStaticDir /* && gfile.Exists(staticFile) */{
if request.isFileRequest /* && gfile.Exists(staticFile) */{
// 静态文件
s.serveFile(request, staticFile)
} else {
@ -115,6 +133,31 @@ func (s *Server)handleRequest(w http.ResponseWriter, r *http.Request) {
s.callHookHandler(HOOK_AFTER_OUTPUT, request)
}
// 查找静态文件的绝对路径
func (s *Server) searchStaticFile(uri string) (filePath string, isDir bool) {
// 优先查找URI映射
if len(s.config.StaticPaths) > 0 {
for _, item := range s.config.StaticPaths {
if len(uri) >= len(item.prefix) && strings.EqualFold(item.prefix, uri[0 : len(item.prefix)]) {
// 防止类似 /static/style 映射到 /static/style.css 的情况
if len(uri) > len(item.prefix) && uri[len(item.prefix)] != '/' {
continue
}
return gspath.Search(item.path, uri[len(item.prefix):], s.config.IndexFiles...)
}
}
}
// 其次查找root和search path
if len(s.config.SearchPaths) > 0 {
for _, path := range s.config.SearchPaths {
if filePath, isDir = gspath.Search(path, uri, s.config.IndexFiles...); filePath != "" {
return filePath, isDir
}
}
}
return "", false
}
// 初始化控制器
func (s *Server) callServeHandler(h *handlerItem, r *Request) {
defer func() {
@ -176,9 +219,15 @@ func (s *Server)listDir(r *Request, f http.File) {
r.Response.Header().Set("Content-Type", "text/html; charset=utf-8")
r.Response.Write("<pre>\n")
if r.URL.Path != "/" {
r.Response.Write(fmt.Sprint("<a href=\"..\">..</a>\n"))
}
for _, file := range files {
name := file.Name()
r.Response.Write(fmt.Sprintf("<a href=\"%s/%s\">%s</a>\n", r.URL.Path, name, ghtml.SpecialChars(name)))
if file.IsDir() {
name += "/"
}
r.Response.Write(fmt.Sprintf("<a href=\"%s\">%s</a>\n", name, ghtml.SpecialChars(name)))
}
r.Response.Write("</pre>\n")
}

View File

@ -8,14 +8,14 @@
package ghttp
import (
"errors"
"strings"
"container/list"
"errors"
"fmt"
"gitee.com/johng/gf/g/os/glog"
"gitee.com/johng/gf/g/util/gregex"
"gitee.com/johng/gf/g/util/gstr"
"gitee.com/johng/gf/g/os/glog"
"fmt"
"runtime"
"strings"
)
@ -55,9 +55,9 @@ func (s *Server) getHandlerRegisterCallerLine(handler *handlerItem) string {
}
// 路由注册处理方法。
// 如果带有hook参数表示是回调注册方法否则为普通路由执行方法。
// 如果带有hook参数表示是回调注册方法; 否则为普通路由执行方法。
func (s *Server) setHandler(pattern string, handler *handlerItem, hook ... string) (resultErr error) {
// Web Server正运行时无法动态注册路由方法
// Web Server正运行时无法动态注册路由方法
if s.Status() == SERVER_STATUS_RUNNING {
return errors.New("cannot bind handler while server running")
}
@ -69,22 +69,27 @@ func (s *Server) setHandler(pattern string, handler *handlerItem, hook ... strin
if err != nil {
return errors.New("invalid pattern")
}
regkey := s.hookHandlerKey(hookName, method, uri, domain)
// 注册地址记录及重复注册判断
regkey := s.handlerKey(hookName, method, uri, domain)
caller := s.getHandlerRegisterCallerLine(handler)
if item, ok := s.routesMap[regkey]; ok {
s := fmt.Sprintf(`duplicated route registry "%s", already registered in %s`, pattern, item.file)
glog.Errorfln(s)
return errors.New(s)
} else {
defer func() {
if resultErr == nil {
s.routesMap[regkey] = registeredRouteItem{
file : caller,
handler : handler,
}
}
}()
if len(hook) == 0 {
if item, ok := s.routesMap[regkey]; ok {
s := fmt.Sprintf(`duplicated route registry "%s", already registered in %s`, pattern, item[0].file)
glog.Errorfln(s)
return errors.New(s)
}
}
defer func() {
if resultErr == nil {
if _, ok := s.routesMap[regkey]; !ok {
s.routesMap[regkey] = make([]registeredRouteItem, 0)
}
s.routesMap[regkey] = append(s.routesMap[regkey], registeredRouteItem {
file : caller,
handler : handler,
})
}
}()
// 路由对象
handler.router = &Router {
@ -156,14 +161,14 @@ func (s *Server) setHandler(pattern string, handler *handlerItem, hook ... strin
}
}
}
// 得到的lists是该路由规则一路匹配下来相关的模糊匹配链表(注意不是这棵树所有的链表)
// 从头开始遍历每个节点的模糊匹配链表,将该路由项插入进去(按照优先级高的放在前面)
// 上面循环后得到的lists是该路由规则一路匹配下来相关的模糊匹配链表(注意不是这棵树所有的链表)
// 下面从头开始遍历每个节点的模糊匹配链表,将该路由项插入进去(按照优先级高的放在lists链表的前面)
item := (*handlerItem)(nil)
for _, l := range lists {
pushed := false
for e := l.Front(); e != nil; e = e.Next() {
item = e.Value.(*handlerItem)
// 判断是否已存在相同的路由注册项,是则进行替换
// 判断是否已存在相同的路由注册项,(如果不是hook注册)是则进行替换
if len(hookName) == 0 {
if strings.EqualFold(handler.router.Domain, item.router.Domain) &&
strings.EqualFold(handler.router.Method, item.router.Method) &&
@ -173,6 +178,7 @@ func (s *Server) setHandler(pattern string, handler *handlerItem, hook ... strin
break
}
}
// 如果路由注册项不相等,那么判断优先级,决定插入顺序
if s.compareRouterPriority(handler.router, item.router) {
l.InsertBefore(handler, e)
pushed = true
@ -189,10 +195,12 @@ func (s *Server) setHandler(pattern string, handler *handlerItem, hook ... strin
}
// 对比两个handlerItem的优先级需要非常注意的是注意新老对比项的参数先后顺序。
// 返回值true表示newRouter优先级比oldRouter高会被添加链表中oldRouter的前面否则后面。
// 优先级比较规则:
// 1、层级越深优先级越高(对比/数量)
// 2、模糊规则优先级{xxx} > :xxx > *xxx
func (s *Server) compareRouterPriority(newRouter, oldRouter *Router) bool {
// 优先比较层级,层级越深优先级越高
if newRouter.Priority > oldRouter.Priority {
return true
}
@ -249,6 +257,9 @@ func (s *Server) compareRouterPriority(newRouter, oldRouter *Router) bool {
if fuzzyCountNameNew < fuzzyCountNameOld {
return false
}
/* 模糊规则数量相等,后续不用再判断*规则的数量比较了 */
// 比较HTTP METHOD更精准的优先级更高
if newRouter.Method != gDEFAULT_METHOD {
return true
@ -256,9 +267,9 @@ func (s *Server) compareRouterPriority(newRouter, oldRouter *Router) bool {
if oldRouter.Method != gDEFAULT_METHOD {
return true
}
// 模糊规则数量相等,后续不用再判断*规则的数量比较了,
// 这种情况下新的规则比旧的规则优先级更高
return true
// 最后新的规则比旧的规则优先级
return false
}
// 将pattern不带method和domain解析成正则表达式匹配以及对应的query字符串

View File

@ -81,11 +81,11 @@ func (s *Server) callHookHandler(hook string, r *Request) {
// 查询请求处理方法, 带缓存机制按照Host、Method、Path进行缓存.
func (s *Server) getHookHandlerWithCache(hook string, r *Request) []*handlerParsedItem {
cacheItems := ([]*handlerParsedItem)(nil)
cacheKey := s.hookHandlerKey(hook, r.Method, r.URL.Path, r.GetHost())
cacheKey := s.handlerKey(hook, r.Method, r.URL.Path, r.GetHost())
if v := s.hooksCache.Get(cacheKey); v == nil {
cacheItems = s.searchHookHandler(r.Method, r.URL.Path, r.GetHost(), hook)
if cacheItems != nil {
s.hooksCache.Set(cacheKey, cacheItems)
s.hooksCache.Set(cacheKey, cacheItems, s.config.RouterCacheExpire*1000)
}
} else {
cacheItems = v.([]*handlerParsedItem)
@ -189,7 +189,7 @@ func (s *Server) searchHookHandler(method, path, domain, hook string) []*handler
}
// 生成hook key如果是hook key那么使用'%'符号分隔
func (s *Server) hookHandlerKey(hook, method, path, domain string) string {
func (s *Server) handlerKey(hook, method, path, domain string) string {
return hook + "%" + s.serveHandlerKey(method, path, domain)
}

View File

@ -21,7 +21,7 @@ func (s *Server) getServeHandlerWithCache(r *Request) *handlerParsedItem {
if v := s.serveCache.Get(cacheKey); v == nil {
cacheItem = s.searchServeHandler(r.Method, r.URL.Path, r.GetHost())
if cacheItem != nil {
s.serveCache.Set(cacheKey, cacheItem)
s.serveCache.Set(cacheKey, cacheItem, s.config.RouterCacheExpire*1000)
}
} else {
cacheItem = v.(*handlerParsedItem)

View File

@ -9,6 +9,7 @@ package ghttp
import (
"gitee.com/johng/gf/g/container/gmap"
"gitee.com/johng/gf/g/container/gvar"
"gitee.com/johng/gf/g/os/gtime"
"gitee.com/johng/gf/g/util/gconv"
"gitee.com/johng/gf/g/util/grand"
@ -30,11 +31,12 @@ func makeSessionId() string {
return strings.ToUpper(strconv.FormatInt(gtime.Nanosecond(), 32) + grand.RandStr(3))
}
// 获取或者生成一个session对象
// 获取或者生成一个session对象(延迟初始化)
func GetSession(r *Request) *Session {
if r.Session != nil {
return r.Session
}
return &Session {
data : gmap.NewStringInterfaceMap(),
server : r.Server,
request : r,
}
}
@ -42,8 +44,9 @@ func GetSession(r *Request) *Session {
// 执行初始化(用于延迟初始化)
func (s *Session) init() {
if len(s.id) == 0 {
s.id = s.request.Cookie.SessionId()
s.data = s.server.sessions.GetOrSetFuncLock(s.id, func() interface{} {
s.id = s.request.Cookie.SessionId()
s.server = s.request.Server
s.data = s.server.sessions.GetOrSetFuncLock(s.id, func() interface{} {
return gmap.NewStringInterfaceMap()
}, s.server.GetSessionMaxAge()).(*gmap.StringInterfaceMap)
}
@ -68,13 +71,13 @@ func (s *Session) Set(key string, value interface{}) {
}
// 批量设置(BatchSet别名)
func (s *Session) Sets (m map[string]interface{}) {
func (s *Session) Sets(m map[string]interface{}) {
s.init()
s.BatchSet(m)
}
// 批量设置
func (s *Session) BatchSet (m map[string]interface{}) {
func (s *Session) BatchSet(m map[string]interface{}) {
s.init()
s.data.BatchSet(m)
}
@ -85,13 +88,21 @@ func (s *Session) Contains (key string) bool {
return s.data.Contains(key)
}
// 获取session
// 获取SESSION
func (s *Session) Get (key string) interface{} {
s.init()
return s.data.Get(key)
}
func (s *Session) GetString (key string) string { return gconv.String(s.Get(key)) }
func (s *Session) GetBool(key string) bool { return gconv.Bool(s.Get(key)) }
// 获取SESSION建议都用该方法获取参数
func (s *Session) GetVar(key string) gvar.VarRead {
s.init()
return gvar.NewRead(s.data.Get(key), false)
}
func (s *Session) GetString (key string) string { return gconv.String(s.Get(key)) }
func (s *Session) GetBool(key string) bool { return gconv.Bool(s.Get(key)) }
func (s *Session) GetInt(key string) int { return gconv.Int(s.Get(key)) }
func (s *Session) GetInt8(key string) int8 { return gconv.Int8(s.Get(key)) }
@ -105,8 +116,8 @@ func (s *Session) GetUint16(key string) uint16 { return gconv.U
func (s *Session) GetUint32(key string) uint32 { return gconv.Uint32(s.Get(key)) }
func (s *Session) GetUint64(key string) uint64 { return gconv.Uint64(s.Get(key)) }
func (s *Session) GetFloat32 (key string) float32 { return gconv.Float32(s.Get(key)) }
func (s *Session) GetFloat64 (key string) float64 { return gconv.Float64(s.Get(key)) }
func (s *Session) GetFloat32 (key string) float32 { return gconv.Float32(s.Get(key)) }
func (s *Session) GetFloat64 (key string) float64 { return gconv.Float64(s.Get(key)) }
func (s *Session) GetBytes(key string) []byte { return gconv.Bytes(s.Get(key)) }
func (s *Session) GetInts(key string) []int { return gconv.Ints(s.Get(key)) }
@ -123,13 +134,13 @@ func (s *Session) GetStruct(key string, objPointer interface{}, attrMapping...ma
}
// 删除session
func (s *Session) Remove (key string) {
func (s *Session) Remove(key string) {
s.init()
s.data.Remove(key)
}
// 清空session
func (s *Session) Clear () {
func (s *Session) Clear() {
s.init()
s.data.Clear()
}

View File

@ -1,17 +1,12 @@
// Copyright 2017 gf Author(https://gitee.com/johng/gf). All Rights Reserved.
// Copyright 2017-2018 gf Author(https://gitee.com/johng/gf). All Rights Reserved.
//
// This Source Code Form is subject to the terms of the MIT License.
// If a copy of the MIT was not distributed with this file,
// You can obtain one at https://gitee.com/johng/gf.
// 单进程高速缓存.
// 并发安全的单进程高速缓存.
package gcache
const (
// 当数据不过期时默认设置的过期属性值相当于math.MaxInt64/1000000
gDEFAULT_MAX_EXPIRE = 9223372036854
)
// 全局缓存管理对象
var cache = New()
@ -66,6 +61,11 @@ func BatchRemove(keys []interface{}) {
cache.BatchRemove(keys)
}
// 返回缓存的所有数据键值对(不包含已过期数据)
func Data() map[interface{}]interface{} {
return cache.Data()
}
// 获得所有的键名,组成数组返回
func Keys() []interface{} {
return cache.Keys()

View File

@ -22,8 +22,7 @@ func New(lruCap...int) *Cache {
c := &Cache {
memCache : newMemCache(lruCap...),
}
go c.autoSyncLoop()
go c.autoClearLoop()
go c.autoLoop()
return c
}

View File

@ -7,33 +7,37 @@
package gcache
import (
"math"
"gitee.com/johng/gf/g/container/gset"
"gitee.com/johng/gf/g/os/gtime"
"sync"
"gitee.com/johng/gf/g/util/gconv"
"gitee.com/johng/gf/g/container/glist"
"gitee.com/johng/gf/g/container/gset"
"gitee.com/johng/gf/g/container/gtype"
"gitee.com/johng/gf/g/os/gtime"
"gitee.com/johng/gf/g/util/gconv"
"math"
"sync"
"time"
)
// 缓存对象
type memCache struct {
dmu sync.RWMutex // data锁(自定义锁的目的是除去键值的断言转换造成的性能损耗)
emu sync.RWMutex // ekmap锁(expire key map)
smu sync.RWMutex // eksets锁(expire key sets)
lru *memCacheLru // LRU缓存限制(只有限定池大小时才启用)
cap int // 控制缓存池大小超过大小则按照LRU算法进行缓存过期处理(默认为0表示不进行限制)
data map[interface{}]memCacheItem // 缓存数据(所有的缓存数据存放哈希表)
ekmap map[interface{}]int64 // 键名对应的分组过期时间(用于相同键名过期时间快速更新)键值为10秒级时间戳
eksets map[int64]*gset.Set // 分组过期时间对应的键名列表(用于自动过期快速删除)键值为10秒级时间戳
eventList *glist.List // 异步处理队列
lruGetList *glist.List // 获取方法的LRU列表
stopChan chan struct{} // 关闭时间通知
dataMu sync.RWMutex
expireTimeMu sync.RWMutex
expireSetMu sync.RWMutex
cap int // 控制缓存池大小超过大小则按照LRU算法进行缓存过期处理(默认为0表示不进行限制)
data map[interface{}]memCacheItem // 缓存数据(所有的缓存数据存放哈希表)
expireTimes map[interface{}]int64 // 键名对应的分组过期时间(用于相同键名过期时间快速更新)键值为10秒级时间戳
expireSets map[int64]*gset.Set // 分组过期时间对应的键名列表(用于自动过期快速删除)键值为10秒级时间戳
lru *memCacheLru // LRU缓存限制(只有限定cap池大小时才启用)
lruGetList *glist.List // Get操作的LRU记录
eventList *glist.List // 异步处理队列
closed *gtype.Bool // 关闭事件通知
}
// 缓存数据项
type memCacheItem struct {
v interface{} // 缓存键值
v interface{} // 键值
e int64 // 过期时间
}
@ -43,81 +47,81 @@ type memCacheEvent struct {
e int64 // 过期时间
}
const (
// 当数据不过期时默认设置的过期属性值相当于math.MaxInt64/1000000
gDEFAULT_MAX_EXPIRE = 9223372036854
)
// 创建底层的缓存对象
func newMemCache(lruCap...int) *memCache {
c := &memCache {
lru : newMemCacheLru(),
data : make(map[interface{}]memCacheItem),
ekmap : make(map[interface{}]int64),
eksets : make(map[int64]*gset.Set),
stopChan : make(chan struct{}),
eventList : glist.New(),
lruGetList : glist.New(),
lruGetList : glist.New(),
data : make(map[interface{}]memCacheItem),
expireTimes : make(map[interface{}]int64),
expireSets : make(map[int64]*gset.Set),
eventList : glist.New(),
closed : gtype.NewBool(),
}
if len(lruCap) > 0 {
c.cap = lruCap[0]
c.lru = newMemCacheLru(c)
c.cap = lruCap[0]
}
return c
}
// 计算过期缓存的键名(将毫秒换算成秒的整数毫秒)
// 计算过期缓存的键名(将毫秒换算成秒的整数毫秒按照10秒进行分组)
func (c *memCache) makeExpireKey(expire int64) int64 {
return int64(math.Ceil(float64(expire/10000) + 1)*10000)
}
// 获取一个过期键名存放Set,如果没有则返回nil
func (c *memCache) getExpireSet(expire int64) *gset.Set {
c.smu.RLock()
if ekset, ok := c.eksets[expire]; ok {
c.smu.RUnlock()
return ekset
}
c.smu.RUnlock()
func (c *memCache) getExpireSet(expire int64) (expireSet *gset.Set) {
c.expireSetMu.RLock()
expireSet, _ = c.expireSets[expire]
c.expireSetMu.RUnlock()
return nil
}
// 获取或者创建一个过期键名存放Set(由于是异步单线程执行因此不会出现创建set时的覆盖问题)
func (c *memCache) getOrNewExpireSet(expire int64) *gset.Set {
if ekset := c.getExpireSet(expire); ekset == nil {
set := gset.New()
c.smu.Lock()
// 二次检索确认
if ekset, ok := c.eksets[expire]; !ok {
c.eksets[expire] = set
func (c *memCache) getOrNewExpireSet(expire int64) (expireSet *gset.Set) {
if expireSet = c.getExpireSet(expire); expireSet == nil {
expireSet = gset.New()
c.expireSetMu.Lock()
// 写锁二次检索确认
if es, ok := c.expireSets[expire]; ok {
expireSet = es
} else {
set = ekset
c.expireSets[expire] = expireSet
}
c.smu.Unlock()
return set
} else {
return ekset
c.expireSetMu.Unlock()
}
return
}
// 设置kv缓存键值对过期时间单位为毫秒expire<=0表示不过期
func (c *memCache) Set(key interface{}, value interface{}, expire int) {
expireTimestamp := c.getInternalExpire(expire)
c.dmu.Lock()
c.data[key] = memCacheItem{v : value, e : expireTimestamp}
c.dmu.Unlock()
c.eventList.PushBack(memCacheEvent{k : key, e : expireTimestamp})
expireTime := c.getInternalExpire(expire)
c.dataMu.Lock()
c.data[key] = memCacheItem{v : value, e : expireTime}
c.dataMu.Unlock()
c.eventList.PushBack(&memCacheEvent{k : key, e : expireTime})
}
// 设置kv缓存键值对内部会对键名的存在性使用写锁进行二次检索确认如果存在则不再写入返回键名对应的键值。
// 在高并发下有用,防止数据写入的并发逻辑错误。
func (c *memCache) doSetWithLockCheck(key interface{}, value interface{}, expire int) interface{} {
expireTimestamp := c.getInternalExpire(expire)
c.dmu.Lock()
c.dataMu.Lock()
if v, ok := c.data[key]; ok && !v.IsExpired() {
c.dmu.Unlock()
return v
c.dataMu.Unlock()
return v.v
}
if f, ok := value.(func() interface {}); ok {
value = f()
}
c.data[key] = memCacheItem{v : value, e : expireTimestamp}
c.dmu.Unlock()
c.eventList.PushBack(memCacheEvent{k : key, e : expireTimestamp})
c.dataMu.Unlock()
c.eventList.PushBack(&memCacheEvent{k : key, e : expireTimestamp})
return value
}
@ -140,23 +144,23 @@ func (c *memCache) SetIfNotExist(key interface{}, value interface{}, expire int)
}
// 批量设置
func (c *memCache) BatchSet(data map[interface{}]interface{}, expire int) {
expireTimestamp := c.getInternalExpire(expire)
func (c *memCache) BatchSet(data map[interface{}]interface{}, expire int) {
expireTime := c.getInternalExpire(expire)
for k, v := range data {
c.dmu.Lock()
c.data[k] = memCacheItem{v: v, e: expireTimestamp}
c.dmu.Unlock()
c.eventList.PushBack(memCacheEvent{k: k, e: expireTimestamp})
c.dataMu.Lock()
c.data[k] = memCacheItem{v: v, e: expireTime}
c.dataMu.Unlock()
c.eventList.PushBack(&memCacheEvent{k: k, e: expireTime})
}
}
// 获取指定键名的值
func (c *memCache) Get(key interface{}) interface{} {
c.dmu.RLock()
c.dataMu.RLock()
item, ok := c.data[key]
c.dmu.RUnlock()
c.dataMu.RUnlock()
if ok && !item.IsExpired() {
// LRU(Least Recently Used)操作记录
// 增加LRU(Least Recently Used)操作记录
if c.cap > 0 {
c.lruGetList.PushBack(key)
}
@ -199,35 +203,50 @@ func (c *memCache) Contains(key interface{}) bool {
}
// 删除指定键值对,并返回被删除的键值
func (c *memCache) Remove(key interface{}) interface{} {
c.dmu.Lock()
func (c *memCache) Remove(key interface{}) (value interface{}) {
c.dataMu.RLock()
item, ok := c.data[key]
c.dataMu.RUnlock()
if ok {
value = item.v
c.dataMu.Lock()
delete(c.data, key)
c.dataMu.Unlock()
c.eventList.PushBack(&memCacheEvent{k: key, e: gtime.Millisecond() - 1000})
}
c.dmu.Unlock()
return item.v
return
}
// 批量删除键值对,并返回被删除的键值对数据
func (c *memCache) BatchRemove(keys []interface{}) {
c.dmu.Lock()
for _, key := range keys {
delete(c.data, key)
c.Remove(key)
}
c.dmu.Unlock()
}
// 返回缓存的所有数据键值对(不包含已过期数据)
func (c *memCache) Data() map[interface{}]interface{} {
m := make(map[interface{}]interface{})
c.dataMu.RLock()
for k, v := range c.data {
if !v.IsExpired() {
m[k] = v.v
}
}
c.dataMu.RUnlock()
return m
}
// 获得所有的键名,组成数组返回
func (c *memCache) Keys() []interface{} {
keys := make([]interface{}, 0)
c.dmu.RLock()
c.dataMu.RLock()
for k, v := range c.data {
if !v.IsExpired() {
keys = append(keys, k)
}
}
c.dmu.RUnlock()
c.dataMu.RUnlock()
return keys
}
@ -239,129 +258,120 @@ func (c *memCache) KeyStrings() []string {
// 获得所有的值,组成数组返回
func (c *memCache) Values() []interface{} {
values := make([]interface{}, 0)
c.dmu.RLock()
c.dataMu.RLock()
for _, v := range c.data {
if !v.IsExpired() {
values = append(values, v.v)
}
}
c.dmu.RUnlock()
c.dataMu.RUnlock()
return values
}
// 获得缓存对象的键值对数量
func (c *memCache) Size() int {
c.dmu.RLock()
length := len(c.data)
c.dmu.RUnlock()
return length
func (c *memCache) Size() (size int) {
c.dataMu.RLock()
size = len(c.data)
c.dataMu.RUnlock()
return
}
// 删除缓存对象
func (c *memCache) Close() {
close(c.stopChan)
c.lru.Close()
}
// 数据自动同步循环
func (c *memCache) autoSyncLoop() {
newe := int64(0)
for {
select {
case <-c.stopChan:
return
default:
for {
v := c.eventList.PopFront()
if v == nil {
break
}
item := v.(memCacheEvent)
nowm := gtime.Millisecond()
// 如果用户设置的时间比当前时间还小,那么表示要自动清除了,
// 这里赋值一个当前时间-10秒的时间在自动清理的goroutine中会自动检测删除该key
if item.e < nowm {
newe = c.makeExpireKey(nowm) - 10000
} else {
newe = c.makeExpireKey(item.e)
}
// 添加该key到对应的过期集合中
// 注意:这里不需要检查存在性,
// 因为在key过期的时候会和原始的键值对中的过期时间做核对。
c.getOrNewExpireSet(newe).Add(item.k)
// 重新设置对应键名的过期时间
c.emu.Lock()
c.ekmap[item.k] = newe
c.emu.Unlock()
// LRU(Least Recently Used)操作记录
if c.cap > 0 {
c.lru.Push(item.k)
}
}
if c.cap > 0 {
// 优先级高的lru key放后面读取列表
for {
if v := c.lruGetList.PopFront(); v != nil {
c.lru.Push(v)
} else {
break
}
}
}
time.Sleep(10 * time.Second)
}
if c.cap > 0 {
c.lru.Close()
}
c.closed.Set(true)
}
// LRU缓存淘汰处理+自动清理过期键值对
// 每隔10秒清除过去60秒的键值对数据
func (c *memCache) autoClearLoop() {
for {
select {
case <- c.stopChan:
return
default:
// 缓存过期处理
ek := c.makeExpireKey(gtime.Millisecond())
eks := []int64{ek - 10000, ek - 20000, ek - 30000, ek - 40000, ek - 50000, ek - 60000}
for _, v := range eks {
if ekset := c.getExpireSet(v); ekset != nil {
ekset.Iterator(func(v interface{}) bool {
return c.clearByKey(v)
})
}
// 数据处理完之后从集合中删除该时间段
c.smu.Lock()
delete(c.eksets, v)
c.smu.Unlock()
}
// LRU缓存淘汰清理
if c.cap > 0 {
for i := c.Size() - c.cap; i > 0; i-- {
if s := c.lru.Pop(); s != nil {
c.clearByKey(s, true)
}
}
}
time.Sleep(10*time.Second)
}
}
// 数据异步任务循环:
// 1、将事件列表中的数据异步处理并同步结果到expireTimes和expireSets属性中
// 2、清理过期键值对数据
func (c *memCache) autoLoop() {
event := (*memCacheEvent)(nil)
oldExpireTime := int64(0)
newExpireTime := int64(0)
for {
if c.closed.Val() {
return
}
// ========================
// 数据同步处理
// ========================
for {
v := c.eventList.PopFront()
if v == nil {
break
}
event = v.(*memCacheEvent)
// 获得旧的过期时间分组
c.expireTimeMu.RLock()
oldExpireTime = c.expireTimes[event.k]
c.expireTimeMu.RUnlock()
// 计算新的过期时间分组
newExpireTime = c.makeExpireKey(event.e)
if newExpireTime != oldExpireTime {
c.getOrNewExpireSet(newExpireTime).Add(event.k)
if oldExpireTime != 0 {
c.getOrNewExpireSet(oldExpireTime).Remove(event.k)
}
// 重新设置对应键名的过期时间
c.expireTimeMu.Lock()
c.expireTimes[event.k] = newExpireTime
c.expireTimeMu.Unlock()
}
// 写入操作也会增加到LRU(Least Recently Used)操作记录
if c.cap > 0 {
c.lru.Push(event.k)
}
}
// 异步处理读取操作的LRU列表
if c.cap > 0 && c.lruGetList.Len() > 0 {
for {
if v := c.lruGetList.PopFront(); v != nil {
c.lru.Push(v)
} else {
break
}
}
}
// ========================
// 缓存过期处理
// ========================
ek := c.makeExpireKey(gtime.Millisecond())
eks := []int64{ek - 10000, ek - 20000, ek - 30000, ek - 40000, ek - 50000}
for _, expireTime := range eks {
if expireSet := c.getExpireSet(expireTime); expireSet != nil {
// 遍历Set执行数据过期删除
expireSet.Iterator(func(key interface{}) bool {
return c.clearByKey(key)
})
// Set数据处理完之后删除该Set
c.expireSetMu.Lock()
delete(c.expireSets, expireTime)
c.expireSetMu.Unlock()
}
}
// 每间隔1秒批量处理一次
time.Sleep(time.Second)
}
}
// 删除对应键名的缓存数据
func (c *memCache) clearByKey(key interface{}, force...bool) bool {
// 删除缓存数据
c.dmu.Lock()
c.dataMu.Lock()
// 删除核对,真正的过期才删除
if item, ok := c.data[key]; (ok && item.IsExpired()) || (len(force) > 0 && force[0]) {
delete(c.data, key)
}
c.dmu.Unlock()
c.dataMu.Unlock()
// 删除异步处理数据项
c.emu.Lock()
delete(c.ekmap, key)
c.emu.Unlock()
c.expireTimeMu.Lock()
delete(c.expireTimes, key)
c.expireTimeMu.Unlock()
// 删除LRU管理对象中指定键名
c.lru.Remove(key)

View File

@ -7,27 +7,31 @@
package gcache
import (
"fmt"
"container/list"
"fmt"
"gitee.com/johng/gf/g/container/glist"
"gitee.com/johng/gf/g/container/gqueue"
"gitee.com/johng/gf/g/container/gmap"
"gitee.com/johng/gf/g/container/gtype"
"time"
)
// LRU算法实现对象底层双向链表使用了标准库的list.List
type memCacheLru struct {
data *gmap.Map // 记录键名与链表中的位置项指针
list *glist.List // 键名历史记录链表
queue *gqueue.Queue // 事件队列
cache *memCache // 所属Cache对象
data *gmap.Map // 记录键名与链表中的位置项指针
list *glist.List // 键名历史记录链表
rawList *glist.List // 事件列表
closed *gtype.Bool // 是否关闭
}
// 创建LRU管理对象
func newMemCacheLru() *memCacheLru {
func newMemCacheLru(cache *memCache) *memCacheLru {
lru := &memCacheLru {
list : glist.New(),
data : gmap.New(),
queue : gqueue.New(),
cache : cache,
data : gmap.New(),
list : glist.New(),
rawList : glist.New(),
closed : gtype.NewBool(),
}
go lru.StartAutoLoop()
return lru
@ -35,7 +39,7 @@ func newMemCacheLru() *memCacheLru {
// 关闭LRU对象
func (lru *memCacheLru) Close() {
lru.queue.Close()
lru.closed.Set(true)
}
// 删除指定数据项
@ -53,7 +57,7 @@ func (lru *memCacheLru) Size() int {
// 添加LRU数据项
func (lru *memCacheLru) Push(key interface{}) {
lru.queue.Push(key)
lru.rawList.PushBack(key)
}
// 从链表尾删除LRU数据项并返回对应数据
@ -70,20 +74,34 @@ func (lru *memCacheLru) Print() {
for _, v := range lru.list.FrontAll() {
fmt.Printf("%v ", v)
}
fmt.Println()
}
// 异步执行协程将queue中的数据同步到list中
func (lru *memCacheLru) StartAutoLoop() {
for {
if v := lru.queue.Pop(); v != nil {
// 删除对应链表项
if v := lru.data.Get(v); v != nil {
lru.list.Remove(v.(*list.Element))
}
// 将数据插入到链表头,并记录对应的链表项到哈希表中,便于检索
lru.data.Set(v, lru.list.PushFront(v))
} else {
break
if lru.closed.Val() {
return
}
// 数据同步
for {
if v := lru.rawList.PopFront(); v != nil {
// 删除对应链表项
if v := lru.data.Get(v); v != nil {
lru.list.Remove(v.(*list.Element))
}
// 将数据插入到链表头,并记录对应的链表项到哈希表中,便于检索
lru.data.Set(v, lru.list.PushFront(v))
} else {
break
}
}
// 数据清理
for i := lru.Size() - lru.cache.cap; i > 0; i-- {
if s := lru.Pop(); s != nil {
lru.cache.clearByKey(s, true)
}
}
time.Sleep(time.Second)
}
}

View File

@ -9,11 +9,15 @@
package gcfg
import (
"bytes"
"errors"
"fmt"
"gitee.com/johng/gf/g/container/garray"
"gitee.com/johng/gf/g/container/gmap"
"gitee.com/johng/gf/g/container/gtype"
"gitee.com/johng/gf/g/container/gvar"
"gitee.com/johng/gf/g/encoding/gjson"
"gitee.com/johng/gf/g/os/gfile"
"gitee.com/johng/gf/g/os/gfsnotify"
"gitee.com/johng/gf/g/os/glog"
"gitee.com/johng/gf/g/os/gspath"
@ -26,7 +30,7 @@ const (
// 配置管理对象
type Config struct {
name *gtype.String // 默认配置文件名称
paths *gspath.SPath // 搜索目录路径
paths *garray.StringArray // 搜索目录路径
jsons *gmap.StringInterfaceMap // 配置文件对象
vc *gtype.Bool // 层级检索是否执行分隔符冲突检测(默认为false检测会比较影响检索效率)
}
@ -39,33 +43,54 @@ func New(path string, file...string) *Config {
}
c := &Config {
name : gtype.NewString(name),
paths : gspath.New(),
paths : garray.NewStringArray(0, 1),
jsons : gmap.NewStringInterfaceMap(),
vc : gtype.NewBool(),
}
c.SetPath(path)
if len(path) > 0 {
c.SetPath(path)
}
return c
}
// 判断从哪个配置文件中获取内容,返回配置文件的绝对路径
func (c *Config) filePath(file...string) string {
func (c *Config) filePath(file...string) (path string) {
name := c.name.Val()
if len(file) > 0 {
name = file[0]
}
path, _ := c.paths.Search(name)
c.paths.RLockFunc(func(array []string) {
for _, v := range array {
if path, _ = gspath.Search(v, name); path != "" {
break
}
}
})
if path == "" {
buffer := bytes.NewBuffer(nil)
buffer.WriteString(fmt.Sprintf("[gcfg] cannot find config file \"%s\" in following paths:", name))
c.paths.RLockFunc(func(array []string) {
for k, v := range array {
buffer.WriteString(fmt.Sprintf("\n%d. %s",k + 1, v))
}
})
glog.Error(buffer.String())
}
return path
}
// 设置配置管理器的配置文件存放目录绝对路径
func (c *Config) SetPath(path string) error {
if rp, err := c.paths.Set(path); err != nil {
glog.Error("gcfg.SetPath failed:", err.Error())
realPath := gfile.RealPath(path)
if realPath == "" {
err := errors.New(fmt.Sprintf(`path "%s" does not exist`, path))
glog.Error(fmt.Sprintf(`[gcfg] SetPath failed: %s`, err.Error()))
return err
} else {
c.jsons.Clear()
glog.Debug("gcfg.SetPath:", rp)
}
c.jsons.Clear()
c.paths.Clear()
c.paths.Append(realPath)
glog.Debug("[gcfg] SetPath:", realPath)
return nil
}
@ -78,44 +103,44 @@ func (c *Config) SetViolenceCheck(check bool) {
// 添加配置管理器的配置文件搜索路径
func (c *Config) AddPath(path string) error {
if rp, err := c.paths.Add(path); err != nil {
glog.Debug("gcfg.AddPath failed:", err.Error())
realPath := gfile.RealPath(path)
if realPath == "" {
err := errors.New(fmt.Sprintf(`path "%s" does not exist`, path))
glog.Error(fmt.Sprintf(`[gcfg] AddPath failed: %s`, err.Error()))
return err
} else {
glog.Debug("gcfg.AddPath:", rp)
}
c.paths.Append(realPath)
glog.Debug("[gcfg] AddPath:", realPath)
return nil
}
// 获取指定文件的绝对路径,默认获取默认的配置文件路径
func (c *Config) GetFilePath(file...string) string {
name := c.name.Val()
if len(file) > 0 {
name = file[0]
}
path, _ := c.paths.Search(name)
return path
return c.filePath(file...)
}
// 设置配置管理对象的默认文件名称
func (c *Config) SetFileName(name string) {
glog.Debug("gcfg.SetFileName:", name)
glog.Debug("[gcfg] SetFileName:", name)
c.name.Set(name)
}
// 添加配置文件到配置管理器中,第二个参数为非必须,如果不输入表示添加进入默认的配置名称中
func (c *Config) getJson(file...string) *gjson.Json {
fpath := c.filePath(file...)
if r := c.jsons.Get(fpath); r != nil {
filePath := c.filePath(file...)
if filePath == "" {
return nil
}
if r := c.jsons.Get(filePath); r != nil {
return r.(*gjson.Json)
}
if j, err := gjson.Load(fpath); err == nil {
if j, err := gjson.Load(filePath); err == nil {
j.SetViolenceCheck(c.vc.Val())
c.addMonitor(fpath)
c.jsons.Set(fpath, j)
c.addMonitor(filePath)
c.jsons.Set(filePath, j)
return j
} else {
glog.Errorfln(`gcfg.Load config file "%s" failed: %s`, fpath, err.Error())
glog.Errorfln(`[gcfg] Load config file "%s" failed: %s`, filePath, err.Error())
}
return nil
}
@ -129,11 +154,11 @@ func (c *Config) Get(pattern string, file...string) interface{} {
}
// 获得配置项,返回动态变量
func (c *Config) GetVar(pattern string, file...string) *gvar.Var {
func (c *Config) GetVar(pattern string, file...string) gvar.VarRead {
if j := c.getJson(file...); j != nil {
return gvar.New(j.Get(pattern))
return gvar.New(j.Get(pattern), false)
}
return nil
return gvar.New(nil, false)
}
// 获得一个键值对关联数组/哈希表,方便操作,不需要自己做类型转换

View File

@ -108,10 +108,9 @@ func IsDir(path string) bool {
return s.IsDir()
}
// 获取当前工作目录
// 获取当前工作目录(SelfDir()方法的别名)
func Pwd() string {
pwd, _ := filepath.Abs(filepath.Dir(os.Args[0]))
return pwd
return SelfDir()
}
// 判断所给路径是否为文件

View File

@ -151,10 +151,6 @@ func Fatalf(format string, v ...interface{}) {
logger.Fatalf(format, v ...)
}
func Fatalln(v ...interface{}) {
logger.Fatalln(v ...)
}
func Fatalfln(format string, v ...interface{}) {
logger.Fatalfln(format, v ...)
}
@ -167,10 +163,6 @@ func Panicf(format string, v ...interface{}) {
logger.Panicf(format, v ...)
}
func Panicln(v ...interface{}) {
logger.Panicln(v ...)
}
func Panicfln(format string, v ...interface{}) {
logger.Panicfln(format, v ...)
}

View File

@ -295,46 +295,35 @@ func (l *Logger) Printfln(format string, v ...interface{}) {
}
func (l *Logger) Fatal(v ...interface{}) {
l.errPrint(fmt.Sprintln(v...))
l.errPrint("[FATA] " + fmt.Sprintln(v...))
os.Exit(1)
}
func (l *Logger) Fatalf(format string, v ...interface{}) {
l.errPrint(fmt.Sprintf(format, v...))
os.Exit(1)
}
func (l *Logger) Fatalln(v ...interface{}) {
l.errPrint(fmt.Sprintln(v...))
l.errPrint("[FATA] " + fmt.Sprintf(format, v...))
os.Exit(1)
}
func (l *Logger) Fatalfln(format string, v ...interface{}) {
l.errPrint(fmt.Sprintf(format + ln, v...))
l.errPrint("[FATA] " + fmt.Sprintf(format + ln, v...))
os.Exit(1)
}
func (l *Logger) Panic(v ...interface{}) {
s := fmt.Sprintln(v...)
l.errPrint(s)
l.errPrint("[PANI] " + s)
panic(s)
}
func (l *Logger) Panicf(format string, v ...interface{}) {
s := fmt.Sprintf(format, v...)
l.errPrint(s)
panic(s)
}
func (l *Logger) Panicln(v ...interface{}) {
s := fmt.Sprintln(v...)
l.errPrint(s)
l.errPrint("[PANI] " + s)
panic(s)
}
func (l *Logger) Panicfln(format string, v ...interface{}) {
s := fmt.Sprintf(format + ln, v...)
l.errPrint(s)
l.errPrint("[PANI] " + s)
panic(s)
}

View File

@ -23,24 +23,46 @@ import (
// 文件目录搜索管理对象
type SPath struct {
paths *garray.StringArray // 搜索路径,按照优先级进行排序
cache *gmap.StringInterfaceMap // 搜索结果缓存map
paths *garray.StringArray // 搜索路径,按照优先级进行排序
cache *gmap.StringStringMap // 搜索结果缓存map
}
// 文件搜索缓存项
type SPathCacheItem struct {
path string // 文件/目录绝对路径
isDir bool // 是否目录
path string // 文件/目录绝对路径
isDir bool // 是否目录
}
var (
// 单个目录路径对应的SPath对象指针用于路径检索对象复用
pathsMap = gmap.NewStringInterfaceMap()
)
// 创建一个搜索对象
func New () *SPath {
return &SPath {
paths : garray.NewStringArray(0, 2),
cache : gmap.NewStringInterfaceMap(),
func New(path...string) *SPath {
sp := &SPath {
paths : garray.NewStringArray(0, 1),
cache : gmap.NewStringStringMap(),
}
if len(path) > 0 {
sp.Add(path[0])
}
return sp
}
// 创建/获取一个单例的搜索对象, root必须为目录的绝对路径
func Get(root string) *SPath {
return pathsMap.GetOrSetFuncLock(root, func() interface{} {
return New(root)
}).(*SPath)
}
// 检索root目录(必须为绝对路径)下面的name文件的绝对路径indexFiles用于指定当检索到的结果为目录时同时检索是否存在这些indexFiles文件
func Search(root string, name string, indexFiles...string) (filePath string, isDir bool) {
return Get(root).Search(name, indexFiles...)
}
// 设置搜索路径,只保留当前设置项,其他搜索路径被清空
func (sp *SPath) Set(path string) (realPath string, err error) {
realPath = gfile.RealPath(path)
@ -53,9 +75,6 @@ func (sp *SPath) Set(path string) (realPath string, err error) {
if realPath == "" {
return realPath, errors.New(fmt.Sprintf(`path "%s" does not exist`, path))
}
if realPath == "" {
return realPath, errors.New("invalid path:" + path)
}
// 设置的搜索路径必须为目录
if gfile.IsDir(realPath) {
realPath = strings.TrimRight(realPath, gfile.Separator)
@ -66,6 +85,7 @@ func (sp *SPath) Set(path string) (realPath string, err error) {
}
sp.paths.Clear()
sp.cache.Clear()
sp.paths.Append(realPath)
sp.updateCacheByPath(realPath)
sp.addMonitorByPath(realPath)
@ -87,9 +107,6 @@ func (sp *SPath) Add(path string) (realPath string, err error) {
if realPath == "" {
return realPath, errors.New(fmt.Sprintf(`path "%s" does not exist`, path))
}
if realPath == "" {
return realPath, errors.New("invalid path:" + path)
}
// 添加的搜索路径必须为目录
if gfile.IsDir(realPath) {
// 如果已经添加则不再添加
@ -108,24 +125,22 @@ func (sp *SPath) Add(path string) (realPath string, err error) {
// 给定的name只是相对文件路径找不到该文件时返回空字符串;
// 当给定indexFiles时如果name时一个目录那么会进一步检索其下对应的indexFiles文件是否存在存在则返回indexFile绝对路径
// 否则返回name目录绝对路径。
func (sp *SPath) Search(name string, indexFiles...string) (path string, isDir bool) {
func (sp *SPath) Search(name string, indexFiles...string) (filePath string, isDir bool) {
name = sp.formatCacheName(name)
if v := sp.cache.Get(name); v != nil {
item := v.(*SPathCacheItem)
if len(indexFiles) > 0 && item.isDir {
if v := sp.cache.Get(name); v != "" {
filePath, isDir = sp.parseCacheValue(v)
if len(indexFiles) > 0 && isDir {
if name == "/" {
name = ""
}
for _, file := range indexFiles {
if v := sp.cache.Get(name + "/" + file); v != nil {
item := v.(*SPathCacheItem)
return item.path, item.isDir
if v := sp.cache.Get(name + "/" + file); v != "" {
return sp.parseCacheValue(v)
}
}
}
return item.path, item.isDir
}
return "", false
return
}
// 从搜索路径中移除指定的文件,这样该文件无法给搜索。
@ -143,6 +158,11 @@ func (sp *SPath) Remove(path string) {
}
}
// 返回当前对象搜索目录路径列表
func (sp *SPath) Paths() []string {
return sp.paths.Slice()
}
// 返回当前对象缓存的所有路径列表
func (sp *SPath) AllPaths() []string {
paths := sp.cache.Keys()
@ -164,35 +184,45 @@ func (sp *SPath) updateCacheByPath(path string) {
// 格式化name返回符合规范的缓存名称分隔符号统一为'/',且前缀必须以'/'开头(类似HTTP URI).
func (sp *SPath) formatCacheName(name string) string {
name = strings.Trim(name, "./")
if runtime.GOOS != "linux" {
name = gstr.Replace(name, "\\", "/")
}
return "/" + name
return "/" + strings.Trim(name, "./")
}
// 根据path计算出对应的缓存name
func (sp *SPath) nameFromPath(filePath, dirPath string) string {
name := gstr.Replace(filePath, dirPath, "")
// 根据path计算出对应的缓存name, dirPath为检索根目录路径
func (sp *SPath) nameFromPath(filePath, rootPath string) string {
name := gstr.Replace(filePath, rootPath, "")
name = sp.formatCacheName(name)
return name
}
// 按照一定数据结构生成缓存的数据项字符串
func (sp *SPath) makeCacheValue(filePath string, isDir bool) string {
if isDir {
return filePath + "_D_"
}
return filePath + "_F_"
}
// 按照一定数据结构解析数据项字符串
func (sp *SPath) parseCacheValue(value string) (filePath string, isDir bool) {
if value[len(value) - 2 : len(value) - 1][0] == 'F' {
return value[: len(value) - 3], false
}
return value[: len(value) - 3], true
}
// 添加path到缓存中(递归)
func (sp *SPath) addToCache(filePath, dirPath string) {
func (sp *SPath) addToCache(filePath, rootPath string) {
// 首先添加自身
idDir := gfile.IsDir(filePath)
sp.cache.SetIfNotExist(sp.nameFromPath(filePath, dirPath), func() interface{} {
return &SPathCacheItem {
path : filePath,
isDir : idDir,
}
})
// 如果添加的是目录,那么需要递归
sp.cache.SetIfNotExist(sp.nameFromPath(filePath, rootPath), sp.makeCacheValue(filePath, idDir))
// 如果添加的是目录,那么需要递归添加
if idDir {
if files, err := gfile.ScanDir(filePath, "*", true); err == nil {
for _, path := range files {
sp.addToCache(path, dirPath)
sp.cache.SetIfNotExist(sp.nameFromPath(path, rootPath), sp.makeCacheValue(path, gfile.IsDir(path)))
}
}
}

View File

@ -10,32 +10,44 @@ import (
"testing"
)
func BenchmarkSecond(b *testing.B) {
func Benchmark_Second(b *testing.B) {
for i := 0; i < b.N; i++ {
Second()
}
}
func BenchmarkMillisecond(b *testing.B) {
func Benchmark_Millisecond(b *testing.B) {
for i := 0; i < b.N; i++ {
Millisecond()
}
}
func BenchmarkMicrosecond(b *testing.B) {
func Benchmark_Microsecond(b *testing.B) {
for i := 0; i < b.N; i++ {
Microsecond()
}
}
func BenchmarkNanosecond(b *testing.B) {
func Benchmark_Nanosecond(b *testing.B) {
for i := 0; i < b.N; i++ {
Nanosecond()
}
}
func BenchmarkStrToTime(b *testing.B) {
func Benchmark_StrToTime(b *testing.B) {
for i := 0; i < b.N; i++ {
StrToTime("2018-02-09T20:46:17.897Z")
}
}
func Benchmark_ParseTimeFromContent(b *testing.B) {
for i := 0; i < b.N; i++ {
ParseTimeFromContent("2018-02-09T20:46:17.897Z")
}
}
func Benchmark_NewFromTimeStamp(b *testing.B) {
for i := 0; i < b.N; i++ {
NewFromTimeStamp(1542674930)
}
}

View File

@ -8,28 +8,30 @@
package gview
import (
"bytes"
"errors"
"fmt"
"gitee.com/johng/gf"
"gitee.com/johng/gf/g/container/garray"
"gitee.com/johng/gf/g/encoding/ghash"
"gitee.com/johng/gf/g/encoding/ghtml"
"gitee.com/johng/gf/g/encoding/gurl"
"gitee.com/johng/gf/g/os/gfcache"
"gitee.com/johng/gf/g/os/gfile"
"gitee.com/johng/gf/g/os/glog"
"gitee.com/johng/gf/g/os/gspath"
"gitee.com/johng/gf/g/os/gtime"
"gitee.com/johng/gf/g/os/gview/internal/text/template"
"gitee.com/johng/gf/g/util/gconv"
"gitee.com/johng/gf/g/util/gstr"
"strings"
"sync"
"bytes"
"errors"
"text/template"
"gitee.com/johng/gf/g/container/gmap"
"gitee.com/johng/gf/g/encoding/ghash"
"gitee.com/johng/gf/g/util/gconv"
"gitee.com/johng/gf/g/os/gspath"
"gitee.com/johng/gf/g/os/gfcache"
"gitee.com/johng/gf/g/encoding/ghtml"
)
// 视图对象
type View struct {
mu sync.RWMutex
paths *gspath.SPath // 模板查找目录(绝对路径)
paths *garray.StringArray // 模板查找目录(绝对路径)
data map[string]interface{} // 模板变量
funcmap map[string]interface{} // FuncMap
delimiters []string // 模板变量分隔符号
@ -41,16 +43,17 @@ type Params = map[string]interface{}
// 函数映射表
type FuncMap = map[string]interface{}
// 视图表
var viewMap = gmap.NewStringInterfaceMap()
// 默认的视图对象
var viewObj *View
// 初始化默认的视图对象
func checkAndInitDefaultView() {
if viewObj == nil {
viewObj = Get(".")
if gfile.SelfDir() != gfile.TempDir() {
viewObj = New(gfile.SelfDir())
} else {
viewObj = New()
}
}
}
@ -60,26 +63,22 @@ func ParseContent(content string, params Params) ([]byte, error) {
return viewObj.ParseContent(content, params)
}
// 获取或者创建一个视图对象
func Get(path string) *View {
if r := viewMap.Get(path); r != nil {
return r.(*View)
}
v := New(path)
viewMap.Set(path, v)
return v
}
// 生成一个视图对象
func New(path string) *View {
func New(path...string) *View {
view := &View {
paths : gspath.New(),
paths : garray.NewStringArray(0, 1),
data : make(map[string]interface{}),
funcmap : make(map[string]interface{}),
delimiters : make([]string, 2),
}
view.SetPath(path)
if len(path) > 0 && len(path[0]) > 0 {
view.SetPath(path[0])
}
view.SetDelimiters("{{", "}}")
// 内置变量
view.data["GF"] = map[string]interface{} {
"version" : gf.VERSION,
}
// 内置方法
view.BindFunc("text", view.funcText)
view.BindFunc("html", view.funcHtmlEncode)
@ -103,23 +102,28 @@ func New(path string) *View {
// 设置模板目录绝对路径
func (view *View) SetPath(path string) error {
if rp, err := view.paths.Set(path); err != nil {
glog.Error("gview.SetPath failed:", err.Error())
realPath := gfile.RealPath(path)
if realPath == "" {
err := errors.New(fmt.Sprintf(`path "%s" does not exist`, path))
glog.Error(fmt.Sprintf(`[gview] SetPath failed: %s`, err.Error()))
return err
} else {
glog.Debug("gview.SetPath:", rp)
}
view.paths.Clear()
view.paths.Append(realPath)
glog.Debug("[gview] SetPath:", realPath)
return nil
}
// 添加模板目录搜索路径
func (view *View) AddPath(path string) error {
if rp, err := view.paths.Add(path); err != nil {
glog.Error("gview.AddPath failed:", err.Error())
realPath := gfile.RealPath(path)
if realPath == "" {
err := errors.New(fmt.Sprintf(`path "%s" does not exist`, path))
glog.Error(fmt.Sprintf(`[gview] AddPath failed: %s`, err.Error()))
return err
} else {
glog.Debug("gview.AddPath:", rp)
}
view.paths.Append(realPath)
glog.Debug("[gview] AddPath:", realPath)
return nil
}
@ -141,9 +145,24 @@ func (view *View) Assign(key string, value interface{}) {
// 解析模板,返回解析后的内容
func (view *View) Parse(file string, params Params, funcmap...map[string]interface{}) ([]byte, error) {
path, _ := view.paths.Search(file)
path := ""
view.paths.RLockFunc(func(array []string) {
for _, v := range array {
if path, _ = gspath.Search(v, file); path != "" {
break
}
}
})
if path == "" {
return nil, errors.New("tpl \"" + file + "\" not found")
buffer := bytes.NewBuffer(nil)
buffer.WriteString(fmt.Sprintf("[gview] cannot find template file \"%s\" in following paths:", file))
view.paths.RLockFunc(func(array []string) {
for k, v := range array {
buffer.WriteString(fmt.Sprintf("\n%d. %s",k + 1, v))
}
})
glog.Error(buffer.String())
return nil, errors.New(fmt.Sprintf(`tpl "%s" not found`, file))
}
content := gfcache.GetContents(path)
// 执行模板解析互斥锁主要是用于funcmap
@ -276,8 +295,15 @@ func (view *View) funcUrlDecode(url interface{}) string {
}
// 模板内置方法date
func (view *View) funcDate(format string, timestamp interface{}) string {
return gtime.NewFromTimeStamp(gconv.Int64(timestamp)).Format(format)
func (view *View) funcDate(format string, timestamp...interface{}) string {
t := int64(0)
if len(timestamp) > 0 {
t = gconv.Int64(timestamp[0])
}
if t == 0 {
t = gtime.Millisecond()
}
return gtime.NewFromTimeStamp(t).Format(format)
}
// 模板内置方法compare

View File

@ -0,0 +1,449 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
/*
Package template implements data-driven templates for generating textual output.
To generate HTML output, see package html/template, which has the same interface
as this package but automatically secures HTML output against certain attacks.
Templates are executed by applying them to a data structure. Annotations in the
template refer to elements of the data structure (typically a field of a struct
or a key in a map) to control execution and derive values to be displayed.
Execution of the template walks the structure and sets the cursor, represented
by a period '.' and called "dot", to the value at the current location in the
structure as execution proceeds.
The input text for a template is UTF-8-encoded text in any format.
"Actions"--data evaluations or control structures--are delimited by
"{{" and "}}"; all text outside actions is copied to the output unchanged.
Except for raw strings, actions may not span newlines, although comments can.
Once parsed, a template may be executed safely in parallel, although if parallel
executions share a Writer the output may be interleaved.
Here is a trivial example that prints "17 items are made of wool".
type Inventory struct {
Material string
Count uint
}
sweaters := Inventory{"wool", 17}
tmpl, err := template.New("test").Parse("{{.Count}} items are made of {{.Material}}")
if err != nil { panic(err) }
err = tmpl.Execute(os.Stdout, sweaters)
if err != nil { panic(err) }
More intricate examples appear below.
Text and spaces
By default, all text between actions is copied verbatim when the template is
executed. For example, the string " items are made of " in the example above appears
on standard output when the program is run.
However, to aid in formatting template source code, if an action's left delimiter
(by default "{{") is followed immediately by a minus sign and ASCII space character
("{{- "), all trailing white space is trimmed from the immediately preceding text.
Similarly, if the right delimiter ("}}") is preceded by a space and minus sign
(" -}}"), all leading white space is trimmed from the immediately following text.
In these trim markers, the ASCII space must be present; "{{-3}}" parses as an
action containing the number -3.
For instance, when executing the template whose source is
"{{23 -}} < {{- 45}}"
the generated output would be
"23<45"
For this trimming, the definition of white space characters is the same as in Go:
space, horizontal tab, carriage return, and newline.
Actions
Here is the list of actions. "Arguments" and "pipelines" are evaluations of
data, defined in detail in the corresponding sections that follow.
*/
// {{/* a comment */}}
// {{- /* a comment with white space trimmed from preceding and following text */ -}}
// A comment; discarded. May contain newlines.
// Comments do not nest and must start and end at the
// delimiters, as shown here.
/*
{{pipeline}}
The default textual representation (the same as would be
printed by fmt.Print) of the value of the pipeline is copied
to the output.
{{if pipeline}} T1 {{end}}
If the value of the pipeline is empty, no output is generated;
otherwise, T1 is executed. The empty values are false, 0, any
nil pointer or interface value, and any array, slice, map, or
string of length zero.
Dot is unaffected.
{{if pipeline}} T1 {{else}} T0 {{end}}
If the value of the pipeline is empty, T0 is executed;
otherwise, T1 is executed. Dot is unaffected.
{{if pipeline}} T1 {{else if pipeline}} T0 {{end}}
To simplify the appearance of if-else chains, the else action
of an if may include another if directly; the effect is exactly
the same as writing
{{if pipeline}} T1 {{else}}{{if pipeline}} T0 {{end}}{{end}}
{{range pipeline}} T1 {{end}}
The value of the pipeline must be an array, slice, map, or channel.
If the value of the pipeline has length zero, nothing is output;
otherwise, dot is set to the successive elements of the array,
slice, or map and T1 is executed. If the value is a map and the
keys are of basic type with a defined order ("comparable"), the
elements will be visited in sorted key order.
{{range pipeline}} T1 {{else}} T0 {{end}}
The value of the pipeline must be an array, slice, map, or channel.
If the value of the pipeline has length zero, dot is unaffected and
T0 is executed; otherwise, dot is set to the successive elements
of the array, slice, or map and T1 is executed.
{{template "name"}}
The template with the specified name is executed with nil data.
{{template "name" pipeline}}
The template with the specified name is executed with dot set
to the value of the pipeline.
{{block "name" pipeline}} T1 {{end}}
A block is shorthand for defining a template
{{define "name"}} T1 {{end}}
and then executing it in place
{{template "name" pipeline}}
The typical use is to define a set of root templates that are
then customized by redefining the block templates within.
{{with pipeline}} T1 {{end}}
If the value of the pipeline is empty, no output is generated;
otherwise, dot is set to the value of the pipeline and T1 is
executed.
{{with pipeline}} T1 {{else}} T0 {{end}}
If the value of the pipeline is empty, dot is unaffected and T0
is executed; otherwise, dot is set to the value of the pipeline
and T1 is executed.
Arguments
An argument is a simple value, denoted by one of the following.
- A boolean, string, character, integer, floating-point, imaginary
or complex constant in Go syntax. These behave like Go's untyped
constants.
- The keyword nil, representing an untyped Go nil.
- The character '.' (period):
.
The result is the value of dot.
- A variable name, which is a (possibly empty) alphanumeric string
preceded by a dollar sign, such as
$piOver2
or
$
The result is the value of the variable.
Variables are described below.
- The name of a field of the data, which must be a struct, preceded
by a period, such as
.Field
The result is the value of the field. Field invocations may be
chained:
.Field1.Field2
Fields can also be evaluated on variables, including chaining:
$x.Field1.Field2
- The name of a key of the data, which must be a map, preceded
by a period, such as
.Key
The result is the map element value indexed by the key.
Key invocations may be chained and combined with fields to any
depth:
.Field1.Key1.Field2.Key2
Although the key must be an alphanumeric identifier, unlike with
field names they do not need to start with an upper case letter.
Keys can also be evaluated on variables, including chaining:
$x.key1.key2
- The name of a niladic method of the data, preceded by a period,
such as
.Method
The result is the value of invoking the method with dot as the
receiver, dot.Method(). Such a method must have one return value (of
any type) or two return values, the second of which is an error.
If it has two and the returned error is non-nil, execution terminates
and an error is returned to the caller as the value of Execute.
Method invocations may be chained and combined with fields and keys
to any depth:
.Field1.Key1.Method1.Field2.Key2.Method2
Methods can also be evaluated on variables, including chaining:
$x.Method1.Field
- The name of a niladic function, such as
fun
The result is the value of invoking the function, fun(). The return
types and values behave as in methods. Functions and function
names are described below.
- A parenthesized instance of one the above, for grouping. The result
may be accessed by a field or map key invocation.
print (.F1 arg1) (.F2 arg2)
(.StructValuedMethod "arg").Field
Arguments may evaluate to any type; if they are pointers the implementation
automatically indirects to the base type when required.
If an evaluation yields a function value, such as a function-valued
field of a struct, the function is not invoked automatically, but it
can be used as a truth value for an if action and the like. To invoke
it, use the call function, defined below.
Pipelines
A pipeline is a possibly chained sequence of "commands". A command is a simple
value (argument) or a function or method call, possibly with multiple arguments:
Argument
The result is the value of evaluating the argument.
.Method [Argument...]
The method can be alone or the last element of a chain but,
unlike methods in the middle of a chain, it can take arguments.
The result is the value of calling the method with the
arguments:
dot.Method(Argument1, etc.)
functionName [Argument...]
The result is the value of calling the function associated
with the name:
function(Argument1, etc.)
Functions and function names are described below.
A pipeline may be "chained" by separating a sequence of commands with pipeline
characters '|'. In a chained pipeline, the result of each command is
passed as the last argument of the following command. The output of the final
command in the pipeline is the value of the pipeline.
The output of a command will be either one value or two values, the second of
which has type error. If that second value is present and evaluates to
non-nil, execution terminates and the error is returned to the caller of
Execute.
Variables
A pipeline inside an action may initialize a variable to capture the result.
The initialization has syntax
$variable := pipeline
where $variable is the name of the variable. An action that declares a
variable produces no output.
Variables previously declared can also be assigned, using the syntax
$variable = pipeline
If a "range" action initializes a variable, the variable is set to the
successive elements of the iteration. Also, a "range" may declare two
variables, separated by a comma:
range $index, $element := pipeline
in which case $index and $element are set to the successive values of the
array/slice index or map key and element, respectively. Note that if there is
only one variable, it is assigned the element; this is opposite to the
convention in Go range clauses.
A variable's scope extends to the "end" action of the control structure ("if",
"with", or "range") in which it is declared, or to the end of the template if
there is no such control structure. A template invocation does not inherit
variables from the point of its invocation.
When execution begins, $ is set to the data argument passed to Execute, that is,
to the starting value of dot.
Examples
Here are some example one-line templates demonstrating pipelines and variables.
All produce the quoted word "output":
{{"\"output\""}}
A string constant.
{{`"output"`}}
A raw string constant.
{{printf "%q" "output"}}
A function call.
{{"output" | printf "%q"}}
A function call whose final argument comes from the previous
command.
{{printf "%q" (print "out" "put")}}
A parenthesized argument.
{{"put" | printf "%s%s" "out" | printf "%q"}}
A more elaborate call.
{{"output" | printf "%s" | printf "%q"}}
A longer chain.
{{with "output"}}{{printf "%q" .}}{{end}}
A with action using dot.
{{with $x := "output" | printf "%q"}}{{$x}}{{end}}
A with action that creates and uses a variable.
{{with $x := "output"}}{{printf "%q" $x}}{{end}}
A with action that uses the variable in another action.
{{with $x := "output"}}{{$x | printf "%q"}}{{end}}
The same, but pipelined.
Functions
During execution functions are found in two function maps: first in the
template, then in the global function map. By default, no functions are defined
in the template but the Funcs method can be used to add them.
Predefined global functions are named as follows.
and
Returns the boolean AND of its arguments by returning the
first empty argument or the last argument, that is,
"and x y" behaves as "if x then y else x". All the
arguments are evaluated.
call
Returns the result of calling the first argument, which
must be a function, with the remaining arguments as parameters.
Thus "call .X.Y 1 2" is, in Go notation, dot.X.Y(1, 2) where
Y is a func-valued field, map entry, or the like.
The first argument must be the result of an evaluation
that yields a value of function type (as distinct from
a predefined function such as print). The function must
return either one or two result values, the second of which
is of type error. If the arguments don't match the function
or the returned error value is non-nil, execution stops.
html
Returns the escaped HTML equivalent of the textual
representation of its arguments. This function is unavailable
in html/template, with a few exceptions.
index
Returns the result of indexing its first argument by the
following arguments. Thus "index x 1 2 3" is, in Go syntax,
x[1][2][3]. Each indexed item must be a map, slice, or array.
js
Returns the escaped JavaScript equivalent of the textual
representation of its arguments.
len
Returns the integer length of its argument.
not
Returns the boolean negation of its single argument.
or
Returns the boolean OR of its arguments by returning the
first non-empty argument or the last argument, that is,
"or x y" behaves as "if x then x else y". All the
arguments are evaluated.
print
An alias for fmt.Sprint
printf
An alias for fmt.Sprintf
println
An alias for fmt.Sprintln
urlquery
Returns the escaped value of the textual representation of
its arguments in a form suitable for embedding in a URL query.
This function is unavailable in html/template, with a few
exceptions.
The boolean functions take any zero value to be false and a non-zero
value to be true.
There is also a set of binary comparison operators defined as
functions:
eq
Returns the boolean truth of arg1 == arg2
ne
Returns the boolean truth of arg1 != arg2
lt
Returns the boolean truth of arg1 < arg2
le
Returns the boolean truth of arg1 <= arg2
gt
Returns the boolean truth of arg1 > arg2
ge
Returns the boolean truth of arg1 >= arg2
For simpler multi-way equality tests, eq (only) accepts two or more
arguments and compares the second and subsequent to the first,
returning in effect
arg1==arg2 || arg1==arg3 || arg1==arg4 ...
(Unlike with || in Go, however, eq is a function call and all the
arguments will be evaluated.)
The comparison functions work on basic types only (or named basic
types, such as "type Celsius float32"). They implement the Go rules
for comparison of values, except that size and exact type are
ignored, so any integer value, signed or unsigned, may be compared
with any other integer value. (The arithmetic value is compared,
not the bit pattern, so all negative integers are less than all
unsigned integers.) However, as usual, one may not compare an int
with a float32 and so on.
Associated templates
Each template is named by a string specified when it is created. Also, each
template is associated with zero or more other templates that it may invoke by
name; such associations are transitive and form a name space of templates.
A template may use a template invocation to instantiate another associated
template; see the explanation of the "template" action above. The name must be
that of a template associated with the template that contains the invocation.
Nested template definitions
When parsing a template, another template may be defined and associated with the
template being parsed. Template definitions must appear at the top level of the
template, much like global variables in a Go program.
The syntax of such definitions is to surround each template declaration with a
"define" and "end" action.
The define action names the template being created by providing a string
constant. Here is a simple example:
`{{define "T1"}}ONE{{end}}
{{define "T2"}}TWO{{end}}
{{define "T3"}}{{template "T1"}} {{template "T2"}}{{end}}
{{template "T3"}}`
This defines two templates, T1 and T2, and a third T3 that invokes the other two
when it is executed. Finally it invokes T3. If executed this template will
produce the text
ONE TWO
By construction, a template may reside in only one association. If it's
necessary to have a template addressable from multiple associations, the
template definition must be parsed multiple times to create distinct *Template
values, or must be copied with the Clone or AddParseTree method.
Parse may be called multiple times to assemble the various associated templates;
see the ParseFiles and ParseGlob functions and methods for simple ways to parse
related templates stored in files.
A template may be executed directly or through ExecuteTemplate, which executes
an associated template identified by name. To invoke our example above, we
might write,
err := tmpl.Execute(os.Stdout, "no data needed")
if err != nil {
log.Fatalf("execution failed: %s", err)
}
or to invoke a particular template explicitly by name,
err := tmpl.ExecuteTemplate(os.Stdout, "T2", "no data needed")
if err != nil {
log.Fatalf("execution failed: %s", err)
}
*/
package template

View File

@ -0,0 +1,986 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package template
import (
"bytes"
"fmt"
"io"
"reflect"
"runtime"
"sort"
"strings"
"gitee.com/johng/gf/g/os/gview/internal/text/template/parse"
)
// maxExecDepth specifies the maximum stack depth of templates within
// templates. This limit is only practically reached by accidentally
// recursive template invocations. This limit allows us to return
// an error instead of triggering a stack overflow.
var maxExecDepth = initMaxExecDepth()
func initMaxExecDepth() int {
if runtime.GOARCH == "wasm" {
return 1000
}
return 100000
}
// state represents the state of an execution. It's not part of the
// template so that multiple executions of the same template
// can execute in parallel.
type state struct {
tmpl *Template
wr io.Writer
node parse.Node // current node, for errors
vars []variable // push-down stack of variable values.
depth int // the height of the stack of executing templates.
}
// variable holds the dynamic value of a variable such as $, $x etc.
type variable struct {
name string
value reflect.Value
}
// push pushes a new variable on the stack.
func (s *state) push(name string, value reflect.Value) {
s.vars = append(s.vars, variable{name, value})
}
// mark returns the length of the variable stack.
func (s *state) mark() int {
return len(s.vars)
}
// pop pops the variable stack up to the mark.
func (s *state) pop(mark int) {
s.vars = s.vars[0:mark]
}
// setVar overwrites the last declared variable with the given name.
// Used by variable assignments.
func (s *state) setVar(name string, value reflect.Value) {
for i := s.mark() - 1; i >= 0; i-- {
if s.vars[i].name == name {
s.vars[i].value = value
return
}
}
s.errorf("undefined variable: %s", name)
}
// setTopVar overwrites the top-nth variable on the stack. Used by range iterations.
func (s *state) setTopVar(n int, value reflect.Value) {
s.vars[len(s.vars)-n].value = value
}
// varValue returns the value of the named variable.
func (s *state) varValue(name string) reflect.Value {
for i := s.mark() - 1; i >= 0; i-- {
if s.vars[i].name == name {
return s.vars[i].value
}
}
s.errorf("undefined variable: %s", name)
return zero
}
var zero reflect.Value
type missingValType struct{}
var missingVal = reflect.ValueOf(missingValType{})
// at marks the state to be on node n, for error reporting.
func (s *state) at(node parse.Node) {
s.node = node
}
// doublePercent returns the string with %'s replaced by %%, if necessary,
// so it can be used safely inside a Printf format string.
func doublePercent(str string) string {
return strings.Replace(str, "%", "%%", -1)
}
// TODO: It would be nice if ExecError was more broken down, but
// the way ErrorContext embeds the template name makes the
// processing too clumsy.
// ExecError is the custom error type returned when Execute has an
// error evaluating its template. (If a write error occurs, the actual
// error is returned; it will not be of type ExecError.)
type ExecError struct {
Name string // Name of template.
Err error // Pre-formatted error.
}
func (e ExecError) Error() string {
return e.Err.Error()
}
// errorf records an ExecError and terminates processing.
func (s *state) errorf(format string, args ...interface{}) {
name := doublePercent(s.tmpl.Name())
if s.node == nil {
format = fmt.Sprintf("template: %s: %s", name, format)
} else {
location, context := s.tmpl.ErrorContext(s.node)
format = fmt.Sprintf("template: %s: executing %q at <%s>: %s", location, name, doublePercent(context), format)
}
panic(ExecError{
Name: s.tmpl.Name(),
Err: fmt.Errorf(format, args...),
})
}
// writeError is the wrapper type used internally when Execute has an
// error writing to its output. We strip the wrapper in errRecover.
// Note that this is not an implementation of error, so it cannot escape
// from the package as an error value.
type writeError struct {
Err error // Original error.
}
func (s *state) writeError(err error) {
panic(writeError{
Err: err,
})
}
// errRecover is the handler that turns panics into returns from the top
// level of Parse.
func errRecover(errp *error) {
e := recover()
if e != nil {
switch err := e.(type) {
case runtime.Error:
panic(e)
case writeError:
*errp = err.Err // Strip the wrapper.
case ExecError:
*errp = err // Keep the wrapper.
default:
panic(e)
}
}
}
// ExecuteTemplate applies the template associated with t that has the given name
// to the specified data object and writes the output to wr.
// If an error occurs executing the template or writing its output,
// execution stops, but partial results may already have been written to
// the output writer.
// A template may be executed safely in parallel, although if parallel
// executions share a Writer the output may be interleaved.
func (t *Template) ExecuteTemplate(wr io.Writer, name string, data interface{}) error {
var tmpl *Template
if t.common != nil {
tmpl = t.tmpl[name]
}
if tmpl == nil {
return fmt.Errorf("template: no template %q associated with template %q", name, t.name)
}
return tmpl.Execute(wr, data)
}
// Execute applies a parsed template to the specified data object,
// and writes the output to wr.
// If an error occurs executing the template or writing its output,
// execution stops, but partial results may already have been written to
// the output writer.
// A template may be executed safely in parallel, although if parallel
// executions share a Writer the output may be interleaved.
//
// If data is a reflect.Value, the template applies to the concrete
// value that the reflect.Value holds, as in fmt.Print.
func (t *Template) Execute(wr io.Writer, data interface{}) error {
return t.execute(wr, data)
}
func (t *Template) execute(wr io.Writer, data interface{}) (err error) {
defer errRecover(&err)
value, ok := data.(reflect.Value)
if !ok {
value = reflect.ValueOf(data)
}
state := &state{
tmpl: t,
wr: wr,
vars: []variable{{"$", value}},
}
if t.Tree == nil || t.Root == nil {
state.errorf("%q is an incomplete or empty template", t.Name())
}
state.walk(value, t.Root)
return
}
// DefinedTemplates returns a string listing the defined templates,
// prefixed by the string "; defined templates are: ". If there are none,
// it returns the empty string. For generating an error message here
// and in html/template.
func (t *Template) DefinedTemplates() string {
if t.common == nil {
return ""
}
var b bytes.Buffer
for name, tmpl := range t.tmpl {
if tmpl.Tree == nil || tmpl.Root == nil {
continue
}
if b.Len() > 0 {
b.WriteString(", ")
}
fmt.Fprintf(&b, "%q", name)
}
var s string
if b.Len() > 0 {
s = "; defined templates are: " + b.String()
}
return s
}
// Walk functions step through the major pieces of the template structure,
// generating output as they go.
func (s *state) walk(dot reflect.Value, node parse.Node) {
s.at(node)
switch node := node.(type) {
case *parse.ActionNode:
// Do not pop variables so they persist until next end.
// Also, if the action declares variables, don't print the result.
val := s.evalPipeline(dot, node.Pipe)
if len(node.Pipe.Decl) == 0 {
s.printValue(node, val)
}
case *parse.IfNode:
s.walkIfOrWith(parse.NodeIf, dot, node.Pipe, node.List, node.ElseList)
case *parse.ListNode:
for _, node := range node.Nodes {
s.walk(dot, node)
}
case *parse.RangeNode:
s.walkRange(dot, node)
case *parse.TemplateNode:
s.walkTemplate(dot, node)
case *parse.TextNode:
if _, err := s.wr.Write(node.Text); err != nil {
s.writeError(err)
}
case *parse.WithNode:
s.walkIfOrWith(parse.NodeWith, dot, node.Pipe, node.List, node.ElseList)
default:
s.errorf("unknown node: %s", node)
}
}
// walkIfOrWith walks an 'if' or 'with' node. The two control structures
// are identical in behavior except that 'with' sets dot.
func (s *state) walkIfOrWith(typ parse.NodeType, dot reflect.Value, pipe *parse.PipeNode, list, elseList *parse.ListNode) {
defer s.pop(s.mark())
val := s.evalPipeline(dot, pipe)
truth, ok := isTrue(val)
if !ok {
s.errorf("if/with can't use %v", val)
}
if truth {
if typ == parse.NodeWith {
s.walk(val, list)
} else {
s.walk(dot, list)
}
} else if elseList != nil {
s.walk(dot, elseList)
}
}
// IsTrue reports whether the value is 'true', in the sense of not the zero of its type,
// and whether the value has a meaningful truth value. This is the definition of
// truth used by if and other such actions.
func IsTrue(val interface{}) (truth, ok bool) {
return isTrue(reflect.ValueOf(val))
}
func isTrue(val reflect.Value) (truth, ok bool) {
if !val.IsValid() {
// Something like var x interface{}, never set. It's a form of nil.
return false, true
}
switch val.Kind() {
case reflect.Array, reflect.Map, reflect.Slice, reflect.String:
truth = val.Len() > 0
case reflect.Bool:
truth = val.Bool()
case reflect.Complex64, reflect.Complex128:
truth = val.Complex() != 0
case reflect.Chan, reflect.Func, reflect.Ptr, reflect.Interface:
truth = !val.IsNil()
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
truth = val.Int() != 0
case reflect.Float32, reflect.Float64:
truth = val.Float() != 0
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
truth = val.Uint() != 0
case reflect.Struct:
truth = true // Struct values are always true.
default:
return
}
return truth, true
}
func (s *state) walkRange(dot reflect.Value, r *parse.RangeNode) {
s.at(r)
defer s.pop(s.mark())
val, _ := indirect(s.evalPipeline(dot, r.Pipe))
// mark top of stack before any variables in the body are pushed.
mark := s.mark()
oneIteration := func(index, elem reflect.Value) {
// Set top var (lexically the second if there are two) to the element.
if len(r.Pipe.Decl) > 0 {
s.setTopVar(1, elem)
}
// Set next var (lexically the first if there are two) to the index.
if len(r.Pipe.Decl) > 1 {
s.setTopVar(2, index)
}
s.walk(elem, r.List)
s.pop(mark)
}
switch val.Kind() {
case reflect.Array, reflect.Slice:
if val.Len() == 0 {
break
}
for i := 0; i < val.Len(); i++ {
oneIteration(reflect.ValueOf(i), val.Index(i))
}
return
case reflect.Map:
if val.Len() == 0 {
break
}
for _, key := range sortKeys(val.MapKeys()) {
oneIteration(key, val.MapIndex(key))
}
return
case reflect.Chan:
if val.IsNil() {
break
}
i := 0
for ; ; i++ {
elem, ok := val.Recv()
if !ok {
break
}
oneIteration(reflect.ValueOf(i), elem)
}
if i == 0 {
break
}
return
case reflect.Invalid:
break // An invalid value is likely a nil map, etc. and acts like an empty map.
default:
s.errorf("range can't iterate over %v", val)
}
if r.ElseList != nil {
s.walk(dot, r.ElseList)
}
}
func (s *state) walkTemplate(dot reflect.Value, t *parse.TemplateNode) {
s.at(t)
tmpl := s.tmpl.tmpl[t.Name]
if tmpl == nil {
s.errorf("template %q not defined", t.Name)
}
if s.depth == maxExecDepth {
s.errorf("exceeded maximum template depth (%v)", maxExecDepth)
}
// Variables declared by the pipeline persist.
dot = s.evalPipeline(dot, t.Pipe)
newState := *s
newState.depth++
newState.tmpl = tmpl
// No dynamic scoping: template invocations inherit no variables.
newState.vars = []variable{{"$", dot}}
newState.walk(dot, tmpl.Root)
}
// Eval functions evaluate pipelines, commands, and their elements and extract
// values from the data structure by examining fields, calling methods, and so on.
// The printing of those values happens only through walk functions.
// evalPipeline returns the value acquired by evaluating a pipeline. If the
// pipeline has a variable declaration, the variable will be pushed on the
// stack. Callers should therefore pop the stack after they are finished
// executing commands depending on the pipeline value.
func (s *state) evalPipeline(dot reflect.Value, pipe *parse.PipeNode) (value reflect.Value) {
if pipe == nil {
return
}
s.at(pipe)
value = missingVal
for _, cmd := range pipe.Cmds {
value = s.evalCommand(dot, cmd, value) // previous value is this one's final arg.
// If the object has type interface{}, dig down one level to the thing inside.
if value.Kind() == reflect.Interface && value.Type().NumMethod() == 0 {
value = reflect.ValueOf(value.Interface()) // lovely!
}
}
for _, variable := range pipe.Decl {
if pipe.IsAssign {
s.setVar(variable.Ident[0], value)
} else {
s.push(variable.Ident[0], value)
}
}
return value
}
func (s *state) notAFunction(args []parse.Node, final reflect.Value) {
if len(args) > 1 || final != missingVal {
s.errorf("can't give argument to non-function %s", args[0])
}
}
func (s *state) evalCommand(dot reflect.Value, cmd *parse.CommandNode, final reflect.Value) reflect.Value {
firstWord := cmd.Args[0]
switch n := firstWord.(type) {
case *parse.FieldNode:
return s.evalFieldNode(dot, n, cmd.Args, final)
case *parse.ChainNode:
return s.evalChainNode(dot, n, cmd.Args, final)
case *parse.IdentifierNode:
// Must be a function.
return s.evalFunction(dot, n, cmd, cmd.Args, final)
case *parse.PipeNode:
// Parenthesized pipeline. The arguments are all inside the pipeline; final is ignored.
return s.evalPipeline(dot, n)
case *parse.VariableNode:
return s.evalVariableNode(dot, n, cmd.Args, final)
}
s.at(firstWord)
s.notAFunction(cmd.Args, final)
switch word := firstWord.(type) {
case *parse.BoolNode:
return reflect.ValueOf(word.True)
case *parse.DotNode:
return dot
case *parse.NilNode:
s.errorf("nil is not a command")
case *parse.NumberNode:
return s.idealConstant(word)
case *parse.StringNode:
return reflect.ValueOf(word.Text)
}
s.errorf("can't evaluate command %q", firstWord)
panic("not reached")
}
// idealConstant is called to return the value of a number in a context where
// we don't know the type. In that case, the syntax of the number tells us
// its type, and we use Go rules to resolve. Note there is no such thing as
// a uint ideal constant in this situation - the value must be of int type.
func (s *state) idealConstant(constant *parse.NumberNode) reflect.Value {
// These are ideal constants but we don't know the type
// and we have no context. (If it was a method argument,
// we'd know what we need.) The syntax guides us to some extent.
s.at(constant)
switch {
case constant.IsComplex:
return reflect.ValueOf(constant.Complex128) // incontrovertible.
case constant.IsFloat && !isHexConstant(constant.Text) && strings.ContainsAny(constant.Text, ".eE"):
return reflect.ValueOf(constant.Float64)
case constant.IsInt:
n := int(constant.Int64)
if int64(n) != constant.Int64 {
s.errorf("%s overflows int", constant.Text)
}
return reflect.ValueOf(n)
case constant.IsUint:
s.errorf("%s overflows int", constant.Text)
}
return zero
}
func isHexConstant(s string) bool {
return len(s) > 2 && s[0] == '0' && (s[1] == 'x' || s[1] == 'X')
}
func (s *state) evalFieldNode(dot reflect.Value, field *parse.FieldNode, args []parse.Node, final reflect.Value) reflect.Value {
s.at(field)
return s.evalFieldChain(dot, dot, field, field.Ident, args, final)
}
func (s *state) evalChainNode(dot reflect.Value, chain *parse.ChainNode, args []parse.Node, final reflect.Value) reflect.Value {
s.at(chain)
if len(chain.Field) == 0 {
s.errorf("internal error: no fields in evalChainNode")
}
if chain.Node.Type() == parse.NodeNil {
s.errorf("indirection through explicit nil in %s", chain)
}
// (pipe).Field1.Field2 has pipe as .Node, fields as .Field. Eval the pipeline, then the fields.
pipe := s.evalArg(dot, nil, chain.Node)
return s.evalFieldChain(dot, pipe, chain, chain.Field, args, final)
}
func (s *state) evalVariableNode(dot reflect.Value, variable *parse.VariableNode, args []parse.Node, final reflect.Value) reflect.Value {
// $x.Field has $x as the first ident, Field as the second. Eval the var, then the fields.
s.at(variable)
value := s.varValue(variable.Ident[0])
if len(variable.Ident) == 1 {
s.notAFunction(args, final)
return value
}
return s.evalFieldChain(dot, value, variable, variable.Ident[1:], args, final)
}
// evalFieldChain evaluates .X.Y.Z possibly followed by arguments.
// dot is the environment in which to evaluate arguments, while
// receiver is the value being walked along the chain.
func (s *state) evalFieldChain(dot, receiver reflect.Value, node parse.Node, ident []string, args []parse.Node, final reflect.Value) reflect.Value {
n := len(ident)
for i := 0; i < n-1; i++ {
receiver = s.evalField(dot, ident[i], node, nil, missingVal, receiver)
}
// Now if it's a method, it gets the arguments.
return s.evalField(dot, ident[n-1], node, args, final, receiver)
}
func (s *state) evalFunction(dot reflect.Value, node *parse.IdentifierNode, cmd parse.Node, args []parse.Node, final reflect.Value) reflect.Value {
s.at(node)
name := node.Ident
function, ok := findFunction(name, s.tmpl)
if !ok {
s.errorf("%q is not a defined function", name)
}
return s.evalCall(dot, function, cmd, name, args, final)
}
// evalField evaluates an expression like (.Field) or (.Field arg1 arg2).
// The 'final' argument represents the return value from the preceding
// value of the pipeline, if any.
func (s *state) evalField(dot reflect.Value, fieldName string, node parse.Node, args []parse.Node, final, receiver reflect.Value) reflect.Value {
if !receiver.IsValid() {
if s.tmpl.option.missingKey == mapError { // Treat invalid value as missing map key.
s.errorf("nil data; no entry for key %q", fieldName)
}
return zero
}
typ := receiver.Type()
receiver, isNil := indirect(receiver)
// Unless it's an interface, need to get to a value of type *T to guarantee
// we see all methods of T and *T.
ptr := receiver
if ptr.Kind() != reflect.Interface && ptr.Kind() != reflect.Ptr && ptr.CanAddr() {
ptr = ptr.Addr()
}
if method := ptr.MethodByName(fieldName); method.IsValid() {
return s.evalCall(dot, method, node, fieldName, args, final)
}
hasArgs := len(args) > 1 || final != missingVal
// It's not a method; must be a field of a struct or an element of a map.
switch receiver.Kind() {
case reflect.Struct:
tField, ok := receiver.Type().FieldByName(fieldName)
if ok {
if isNil {
s.errorf("nil pointer evaluating %s.%s", typ, fieldName)
}
field := receiver.FieldByIndex(tField.Index)
if tField.PkgPath != "" { // field is unexported
s.errorf("%s is an unexported field of struct type %s", fieldName, typ)
}
// If it's a function, we must call it.
if hasArgs {
s.errorf("%s has arguments but cannot be invoked as function", fieldName)
}
return field
}
case reflect.Map:
if isNil {
s.errorf("nil pointer evaluating %s.%s", typ, fieldName)
}
// If it's a map, attempt to use the field name as a key.
nameVal := reflect.ValueOf(fieldName)
if nameVal.Type().AssignableTo(receiver.Type().Key()) {
if hasArgs {
s.errorf("%s is not a method but has arguments", fieldName)
}
result := receiver.MapIndex(nameVal)
if !result.IsValid() {
switch s.tmpl.option.missingKey {
case mapInvalid:
// Just use the invalid value.
case mapZeroValue:
result = reflect.Zero(receiver.Type().Elem())
case mapError:
s.errorf("map has no entry for key %q", fieldName)
}
}
return result
}
}
s.errorf("can't evaluate field %s in type %s", fieldName, typ)
panic("not reached")
}
var (
errorType = reflect.TypeOf((*error)(nil)).Elem()
fmtStringerType = reflect.TypeOf((*fmt.Stringer)(nil)).Elem()
reflectValueType = reflect.TypeOf((*reflect.Value)(nil)).Elem()
)
// evalCall executes a function or method call. If it's a method, fun already has the receiver bound, so
// it looks just like a function call. The arg list, if non-nil, includes (in the manner of the shell), arg[0]
// as the function itself.
func (s *state) evalCall(dot, fun reflect.Value, node parse.Node, name string, args []parse.Node, final reflect.Value) reflect.Value {
if args != nil {
args = args[1:] // Zeroth arg is function name/node; not passed to function.
}
typ := fun.Type()
numIn := len(args)
if final != missingVal {
numIn++
}
numFixed := len(args)
if typ.IsVariadic() {
numFixed = typ.NumIn() - 1 // last arg is the variadic one.
if numIn < numFixed {
s.errorf("wrong number of args for %s: want at least %d got %d", name, typ.NumIn()-1, len(args))
}
} else if numIn != typ.NumIn() {
s.errorf("wrong number of args for %s: want %d got %d", name, typ.NumIn(), numIn)
}
if !goodFunc(typ) {
// TODO: This could still be a confusing error; maybe goodFunc should provide info.
s.errorf("can't call method/function %q with %d results", name, typ.NumOut())
}
// Build the arg list.
argv := make([]reflect.Value, numIn)
// Args must be evaluated. Fixed args first.
i := 0
for ; i < numFixed && i < len(args); i++ {
argv[i] = s.evalArg(dot, typ.In(i), args[i])
}
// Now the ... args.
if typ.IsVariadic() {
argType := typ.In(typ.NumIn() - 1).Elem() // Argument is a slice.
for ; i < len(args); i++ {
argv[i] = s.evalArg(dot, argType, args[i])
}
}
// Add final value if necessary.
if final != missingVal {
t := typ.In(typ.NumIn() - 1)
if typ.IsVariadic() {
if numIn-1 < numFixed {
// The added final argument corresponds to a fixed parameter of the function.
// Validate against the type of the actual parameter.
t = typ.In(numIn - 1)
} else {
// The added final argument corresponds to the variadic part.
// Validate against the type of the elements of the variadic slice.
t = t.Elem()
}
}
argv[i] = s.validateType(final, t)
}
result := fun.Call(argv)
// If we have an error that is not nil, stop execution and return that error to the caller.
if len(result) == 2 && !result[1].IsNil() {
s.at(node)
s.errorf("error calling %s: %s", name, result[1].Interface().(error))
}
v := result[0]
if v.Type() == reflectValueType {
v = v.Interface().(reflect.Value)
}
return v
}
// canBeNil reports whether an untyped nil can be assigned to the type. See reflect.Zero.
func canBeNil(typ reflect.Type) bool {
switch typ.Kind() {
case reflect.Chan, reflect.Func, reflect.Interface, reflect.Map, reflect.Ptr, reflect.Slice:
return true
case reflect.Struct:
return typ == reflectValueType
}
return false
}
// validateType guarantees that the value is valid and assignable to the type.
func (s *state) validateType(value reflect.Value, typ reflect.Type) reflect.Value {
if !value.IsValid() {
if typ == nil {
// An untyped nil interface{}. Accept as a proper nil value.
return reflect.ValueOf(nil)
}
if canBeNil(typ) {
// Like above, but use the zero value of the non-nil type.
return reflect.Zero(typ)
}
s.errorf("invalid value; expected %s", typ)
}
if typ == reflectValueType && value.Type() != typ {
return reflect.ValueOf(value)
}
if typ != nil && !value.Type().AssignableTo(typ) {
if value.Kind() == reflect.Interface && !value.IsNil() {
value = value.Elem()
if value.Type().AssignableTo(typ) {
return value
}
// fallthrough
}
// Does one dereference or indirection work? We could do more, as we
// do with method receivers, but that gets messy and method receivers
// are much more constrained, so it makes more sense there than here.
// Besides, one is almost always all you need.
switch {
case value.Kind() == reflect.Ptr && value.Type().Elem().AssignableTo(typ):
value = value.Elem()
if !value.IsValid() {
s.errorf("dereference of nil pointer of type %s", typ)
}
case reflect.PtrTo(value.Type()).AssignableTo(typ) && value.CanAddr():
value = value.Addr()
default:
s.errorf("wrong type for value; expected %s; got %s", typ, value.Type())
}
}
return value
}
func (s *state) evalArg(dot reflect.Value, typ reflect.Type, n parse.Node) reflect.Value {
s.at(n)
switch arg := n.(type) {
case *parse.DotNode:
return s.validateType(dot, typ)
case *parse.NilNode:
if canBeNil(typ) {
return reflect.Zero(typ)
}
s.errorf("cannot assign nil to %s", typ)
case *parse.FieldNode:
return s.validateType(s.evalFieldNode(dot, arg, []parse.Node{n}, missingVal), typ)
case *parse.VariableNode:
return s.validateType(s.evalVariableNode(dot, arg, nil, missingVal), typ)
case *parse.PipeNode:
return s.validateType(s.evalPipeline(dot, arg), typ)
case *parse.IdentifierNode:
return s.validateType(s.evalFunction(dot, arg, arg, nil, missingVal), typ)
case *parse.ChainNode:
return s.validateType(s.evalChainNode(dot, arg, nil, missingVal), typ)
}
switch typ.Kind() {
case reflect.Bool:
return s.evalBool(typ, n)
case reflect.Complex64, reflect.Complex128:
return s.evalComplex(typ, n)
case reflect.Float32, reflect.Float64:
return s.evalFloat(typ, n)
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
return s.evalInteger(typ, n)
case reflect.Interface:
if typ.NumMethod() == 0 {
return s.evalEmptyInterface(dot, n)
}
case reflect.Struct:
if typ == reflectValueType {
return reflect.ValueOf(s.evalEmptyInterface(dot, n))
}
case reflect.String:
return s.evalString(typ, n)
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
return s.evalUnsignedInteger(typ, n)
}
s.errorf("can't handle %s for arg of type %s", n, typ)
panic("not reached")
}
func (s *state) evalBool(typ reflect.Type, n parse.Node) reflect.Value {
s.at(n)
if n, ok := n.(*parse.BoolNode); ok {
value := reflect.New(typ).Elem()
value.SetBool(n.True)
return value
}
s.errorf("expected bool; found %s", n)
panic("not reached")
}
func (s *state) evalString(typ reflect.Type, n parse.Node) reflect.Value {
s.at(n)
if n, ok := n.(*parse.StringNode); ok {
value := reflect.New(typ).Elem()
value.SetString(n.Text)
return value
}
s.errorf("expected string; found %s", n)
panic("not reached")
}
func (s *state) evalInteger(typ reflect.Type, n parse.Node) reflect.Value {
s.at(n)
if n, ok := n.(*parse.NumberNode); ok && n.IsInt {
value := reflect.New(typ).Elem()
value.SetInt(n.Int64)
return value
}
s.errorf("expected integer; found %s", n)
panic("not reached")
}
func (s *state) evalUnsignedInteger(typ reflect.Type, n parse.Node) reflect.Value {
s.at(n)
if n, ok := n.(*parse.NumberNode); ok && n.IsUint {
value := reflect.New(typ).Elem()
value.SetUint(n.Uint64)
return value
}
s.errorf("expected unsigned integer; found %s", n)
panic("not reached")
}
func (s *state) evalFloat(typ reflect.Type, n parse.Node) reflect.Value {
s.at(n)
if n, ok := n.(*parse.NumberNode); ok && n.IsFloat {
value := reflect.New(typ).Elem()
value.SetFloat(n.Float64)
return value
}
s.errorf("expected float; found %s", n)
panic("not reached")
}
func (s *state) evalComplex(typ reflect.Type, n parse.Node) reflect.Value {
if n, ok := n.(*parse.NumberNode); ok && n.IsComplex {
value := reflect.New(typ).Elem()
value.SetComplex(n.Complex128)
return value
}
s.errorf("expected complex; found %s", n)
panic("not reached")
}
func (s *state) evalEmptyInterface(dot reflect.Value, n parse.Node) reflect.Value {
s.at(n)
switch n := n.(type) {
case *parse.BoolNode:
return reflect.ValueOf(n.True)
case *parse.DotNode:
return dot
case *parse.FieldNode:
return s.evalFieldNode(dot, n, nil, missingVal)
case *parse.IdentifierNode:
return s.evalFunction(dot, n, n, nil, missingVal)
case *parse.NilNode:
// NilNode is handled in evalArg, the only place that calls here.
s.errorf("evalEmptyInterface: nil (can't happen)")
case *parse.NumberNode:
return s.idealConstant(n)
case *parse.StringNode:
return reflect.ValueOf(n.Text)
case *parse.VariableNode:
return s.evalVariableNode(dot, n, nil, missingVal)
case *parse.PipeNode:
return s.evalPipeline(dot, n)
}
s.errorf("can't handle assignment of %s to empty interface argument", n)
panic("not reached")
}
// indirect returns the item at the end of indirection, and a bool to indicate if it's nil.
func indirect(v reflect.Value) (rv reflect.Value, isNil bool) {
for ; v.Kind() == reflect.Ptr || v.Kind() == reflect.Interface; v = v.Elem() {
if v.IsNil() {
return v, true
}
}
return v, false
}
// indirectInterface returns the concrete value in an interface value,
// or else the zero reflect.Value.
// That is, if v represents the interface value x, the result is the same as reflect.ValueOf(x):
// the fact that x was an interface value is forgotten.
func indirectInterface(v reflect.Value) reflect.Value {
if v.Kind() != reflect.Interface {
return v
}
if v.IsNil() {
return reflect.Value{}
}
return v.Elem()
}
// printValue writes the textual representation of the value to the output of
// the template.
func (s *state) printValue(n parse.Node, v reflect.Value) {
s.at(n)
iface, ok := printableValue(v)
if !ok {
s.errorf("can't print %s of type %s", n, v.Type())
}
_, err := fmt.Fprint(s.wr, iface)
if err != nil {
s.writeError(err)
}
}
// printableValue returns the, possibly indirected, interface value inside v that
// is best for a call to formatted printer.
func printableValue(v reflect.Value) (interface{}, bool) {
if v.Kind() == reflect.Ptr {
v, _ = indirect(v) // fmt.Fprint handles nil.
}
if !v.IsValid() {
return "", true
}
if !v.Type().Implements(errorType) && !v.Type().Implements(fmtStringerType) {
if v.CanAddr() && (reflect.PtrTo(v.Type()).Implements(errorType) || reflect.PtrTo(v.Type()).Implements(fmtStringerType)) {
v = v.Addr()
} else {
switch v.Kind() {
case reflect.Chan, reflect.Func:
return nil, false
}
}
}
return v.Interface(), true
}
// sortKeys sorts (if it can) the slice of reflect.Values, which is a slice of map keys.
func sortKeys(v []reflect.Value) []reflect.Value {
if len(v) <= 1 {
return v
}
switch v[0].Kind() {
case reflect.Float32, reflect.Float64:
sort.Slice(v, func(i, j int) bool {
return v[i].Float() < v[j].Float()
})
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
sort.Slice(v, func(i, j int) bool {
return v[i].Int() < v[j].Int()
})
case reflect.String:
sort.Slice(v, func(i, j int) bool {
return v[i].String() < v[j].String()
})
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
sort.Slice(v, func(i, j int) bool {
return v[i].Uint() < v[j].Uint()
})
}
return v
}

View File

@ -0,0 +1,667 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package template
import (
"bytes"
"errors"
"fmt"
"io"
"net/url"
"reflect"
"strings"
"unicode"
"unicode/utf8"
)
// FuncMap is the type of the map defining the mapping from names to functions.
// Each function must have either a single return value, or two return values of
// which the second has type error. In that case, if the second (error)
// return value evaluates to non-nil during execution, execution terminates and
// Execute returns that error.
//
// When template execution invokes a function with an argument list, that list
// must be assignable to the function's parameter types. Functions meant to
// apply to arguments of arbitrary type can use parameters of type interface{} or
// of type reflect.Value. Similarly, functions meant to return a result of arbitrary
// type can return interface{} or reflect.Value.
type FuncMap map[string]interface{}
var builtins = FuncMap{
"and": and,
"call": call,
"html": HTMLEscaper,
"index": index,
"js": JSEscaper,
"len": length,
"not": not,
"or": or,
"print": fmt.Sprint,
"printf": fmt.Sprintf,
"println": fmt.Sprintln,
"urlquery": URLQueryEscaper,
// Comparisons
"eq": eq, // ==
"ge": ge, // >=
"gt": gt, // >
"le": le, // <=
"lt": lt, // <
"ne": ne, // !=
}
var builtinFuncs = createValueFuncs(builtins)
// createValueFuncs turns a FuncMap into a map[string]reflect.Value
func createValueFuncs(funcMap FuncMap) map[string]reflect.Value {
m := make(map[string]reflect.Value)
addValueFuncs(m, funcMap)
return m
}
// addValueFuncs adds to values the functions in funcs, converting them to reflect.Values.
func addValueFuncs(out map[string]reflect.Value, in FuncMap) {
for name, fn := range in {
if !goodName(name) {
panic(fmt.Errorf("function name %s is not a valid identifier", name))
}
v := reflect.ValueOf(fn)
if v.Kind() != reflect.Func {
panic("value for " + name + " not a function")
}
if !goodFunc(v.Type()) {
panic(fmt.Errorf("can't install method/function %q with %d results", name, v.Type().NumOut()))
}
out[name] = v
}
}
// addFuncs adds to values the functions in funcs. It does no checking of the input -
// call addValueFuncs first.
func addFuncs(out, in FuncMap) {
for name, fn := range in {
out[name] = fn
}
}
// goodFunc reports whether the function or method has the right result signature.
func goodFunc(typ reflect.Type) bool {
// We allow functions with 1 result or 2 results where the second is an error.
switch {
case typ.NumOut() == 1:
return true
case typ.NumOut() == 2 && typ.Out(1) == errorType:
return true
}
return false
}
// goodName reports whether the function name is a valid identifier.
func goodName(name string) bool {
if name == "" {
return false
}
for i, r := range name {
switch {
case r == '_':
case i == 0 && !unicode.IsLetter(r):
return false
case !unicode.IsLetter(r) && !unicode.IsDigit(r):
return false
}
}
return true
}
// findFunction looks for a function in the template, and global map.
func findFunction(name string, tmpl *Template) (reflect.Value, bool) {
if tmpl != nil && tmpl.common != nil {
tmpl.muFuncs.RLock()
defer tmpl.muFuncs.RUnlock()
if fn := tmpl.execFuncs[name]; fn.IsValid() {
return fn, true
}
}
if fn := builtinFuncs[name]; fn.IsValid() {
return fn, true
}
return reflect.Value{}, false
}
// prepareArg checks if value can be used as an argument of type argType, and
// converts an invalid value to appropriate zero if possible.
func prepareArg(value reflect.Value, argType reflect.Type) (reflect.Value, error) {
if !value.IsValid() {
if !canBeNil(argType) {
return reflect.Value{}, fmt.Errorf("value is nil; should be of type %s", argType)
}
value = reflect.Zero(argType)
}
if value.Type().AssignableTo(argType) {
return value, nil
}
if intLike(value.Kind()) && intLike(argType.Kind()) && value.Type().ConvertibleTo(argType) {
value = value.Convert(argType)
return value, nil
}
return reflect.Value{}, fmt.Errorf("value has type %s; should be %s", value.Type(), argType)
}
func intLike(typ reflect.Kind) bool {
switch typ {
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
return true
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
return true
}
return false
}
// Indexing.
// index returns the result of indexing its first argument by the following
// arguments. Thus "index x 1 2 3" is, in Go syntax, x[1][2][3]. Each
// indexed item must be a map, slice, or array.
func index(item reflect.Value, indices ...reflect.Value) (reflect.Value, error) {
v := indirectInterface(item)
if !v.IsValid() {
return reflect.Value{}, fmt.Errorf("index of untyped nil")
}
for _, i := range indices {
index := indirectInterface(i)
var isNil bool
if v, isNil = indirect(v); isNil {
return reflect.Value{}, fmt.Errorf("index of nil pointer")
}
switch v.Kind() {
case reflect.Array, reflect.Slice, reflect.String:
var x int64
switch index.Kind() {
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
x = index.Int()
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
x = int64(index.Uint())
case reflect.Invalid:
return reflect.Value{}, fmt.Errorf("cannot index slice/array with nil")
default:
return reflect.Value{}, fmt.Errorf("cannot index slice/array with type %s", index.Type())
}
if x < 0 || x >= int64(v.Len()) {
return reflect.Value{}, fmt.Errorf("index out of range: %d", x)
}
v = v.Index(int(x))
case reflect.Map:
index, err := prepareArg(index, v.Type().Key())
if err != nil {
return reflect.Value{}, err
}
if x := v.MapIndex(index); x.IsValid() {
v = x
} else {
v = reflect.Zero(v.Type().Elem())
}
case reflect.Invalid:
// the loop holds invariant: v.IsValid()
panic("unreachable")
default:
return reflect.Value{}, fmt.Errorf("can't index item of type %s", v.Type())
}
}
return v, nil
}
// Length
// length returns the length of the item, with an error if it has no defined length.
func length(item interface{}) (int, error) {
v := reflect.ValueOf(item)
if !v.IsValid() {
return 0, fmt.Errorf("len of untyped nil")
}
v, isNil := indirect(v)
if isNil {
return 0, fmt.Errorf("len of nil pointer")
}
switch v.Kind() {
case reflect.Array, reflect.Chan, reflect.Map, reflect.Slice, reflect.String:
return v.Len(), nil
}
return 0, fmt.Errorf("len of type %s", v.Type())
}
// Function invocation
// call returns the result of evaluating the first argument as a function.
// The function must return 1 result, or 2 results, the second of which is an error.
func call(fn reflect.Value, args ...reflect.Value) (reflect.Value, error) {
v := indirectInterface(fn)
if !v.IsValid() {
return reflect.Value{}, fmt.Errorf("call of nil")
}
typ := v.Type()
if typ.Kind() != reflect.Func {
return reflect.Value{}, fmt.Errorf("non-function of type %s", typ)
}
if !goodFunc(typ) {
return reflect.Value{}, fmt.Errorf("function called with %d args; should be 1 or 2", typ.NumOut())
}
numIn := typ.NumIn()
var dddType reflect.Type
if typ.IsVariadic() {
if len(args) < numIn-1 {
return reflect.Value{}, fmt.Errorf("wrong number of args: got %d want at least %d", len(args), numIn-1)
}
dddType = typ.In(numIn - 1).Elem()
} else {
if len(args) != numIn {
return reflect.Value{}, fmt.Errorf("wrong number of args: got %d want %d", len(args), numIn)
}
}
argv := make([]reflect.Value, len(args))
for i, arg := range args {
value := indirectInterface(arg)
// Compute the expected type. Clumsy because of variadics.
var argType reflect.Type
if !typ.IsVariadic() || i < numIn-1 {
argType = typ.In(i)
} else {
argType = dddType
}
var err error
if argv[i], err = prepareArg(value, argType); err != nil {
return reflect.Value{}, fmt.Errorf("arg %d: %s", i, err)
}
}
result := v.Call(argv)
if len(result) == 2 && !result[1].IsNil() {
return result[0], result[1].Interface().(error)
}
return result[0], nil
}
// Boolean logic.
func truth(arg reflect.Value) bool {
t, _ := isTrue(indirectInterface(arg))
return t
}
// and computes the Boolean AND of its arguments, returning
// the first false argument it encounters, or the last argument.
func and(arg0 reflect.Value, args ...reflect.Value) reflect.Value {
if !truth(arg0) {
return arg0
}
for i := range args {
arg0 = args[i]
if !truth(arg0) {
break
}
}
return arg0
}
// or computes the Boolean OR of its arguments, returning
// the first true argument it encounters, or the last argument.
func or(arg0 reflect.Value, args ...reflect.Value) reflect.Value {
if truth(arg0) {
return arg0
}
for i := range args {
arg0 = args[i]
if truth(arg0) {
break
}
}
return arg0
}
// not returns the Boolean negation of its argument.
func not(arg reflect.Value) bool {
return !truth(arg)
}
// Comparison.
// TODO: Perhaps allow comparison between signed and unsigned integers.
var (
errBadComparisonType = errors.New("invalid type for comparison")
errBadComparison = errors.New("incompatible types for comparison")
errNoComparison = errors.New("missing argument for comparison")
)
type kind int
const (
invalidKind kind = iota
boolKind
complexKind
intKind
floatKind
stringKind
uintKind
)
func basicKind(v reflect.Value) (kind, error) {
switch v.Kind() {
case reflect.Bool:
return boolKind, nil
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
return intKind, nil
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
return uintKind, nil
case reflect.Float32, reflect.Float64:
return floatKind, nil
case reflect.Complex64, reflect.Complex128:
return complexKind, nil
case reflect.String:
return stringKind, nil
}
return invalidKind, errBadComparisonType
}
// eq evaluates the comparison a == b || a == c || ...
func eq(arg1 reflect.Value, arg2 ...reflect.Value) (bool, error) {
v1 := indirectInterface(arg1)
k1, err := basicKind(v1)
if err != nil {
return false, err
}
if len(arg2) == 0 {
return false, errNoComparison
}
for _, arg := range arg2 {
v2 := indirectInterface(arg)
k2, err := basicKind(v2)
if err != nil {
return false, err
}
truth := false
if k1 != k2 {
// Special case: Can compare integer values regardless of type's sign.
switch {
case k1 == intKind && k2 == uintKind:
truth = v1.Int() >= 0 && uint64(v1.Int()) == v2.Uint()
case k1 == uintKind && k2 == intKind:
truth = v2.Int() >= 0 && v1.Uint() == uint64(v2.Int())
default:
return false, errBadComparison
}
} else {
switch k1 {
case boolKind:
truth = v1.Bool() == v2.Bool()
case complexKind:
truth = v1.Complex() == v2.Complex()
case floatKind:
truth = v1.Float() == v2.Float()
case intKind:
truth = v1.Int() == v2.Int()
case stringKind:
truth = v1.String() == v2.String()
case uintKind:
truth = v1.Uint() == v2.Uint()
default:
panic("invalid kind")
}
}
if truth {
return true, nil
}
}
return false, nil
}
// ne evaluates the comparison a != b.
func ne(arg1, arg2 reflect.Value) (bool, error) {
// != is the inverse of ==.
equal, err := eq(arg1, arg2)
return !equal, err
}
// lt evaluates the comparison a < b.
func lt(arg1, arg2 reflect.Value) (bool, error) {
v1 := indirectInterface(arg1)
k1, err := basicKind(v1)
if err != nil {
return false, err
}
v2 := indirectInterface(arg2)
k2, err := basicKind(v2)
if err != nil {
return false, err
}
truth := false
if k1 != k2 {
// Special case: Can compare integer values regardless of type's sign.
switch {
case k1 == intKind && k2 == uintKind:
truth = v1.Int() < 0 || uint64(v1.Int()) < v2.Uint()
case k1 == uintKind && k2 == intKind:
truth = v2.Int() >= 0 && v1.Uint() < uint64(v2.Int())
default:
return false, errBadComparison
}
} else {
switch k1 {
case boolKind, complexKind:
return false, errBadComparisonType
case floatKind:
truth = v1.Float() < v2.Float()
case intKind:
truth = v1.Int() < v2.Int()
case stringKind:
truth = v1.String() < v2.String()
case uintKind:
truth = v1.Uint() < v2.Uint()
default:
panic("invalid kind")
}
}
return truth, nil
}
// le evaluates the comparison <= b.
func le(arg1, arg2 reflect.Value) (bool, error) {
// <= is < or ==.
lessThan, err := lt(arg1, arg2)
if lessThan || err != nil {
return lessThan, err
}
return eq(arg1, arg2)
}
// gt evaluates the comparison a > b.
func gt(arg1, arg2 reflect.Value) (bool, error) {
// > is the inverse of <=.
lessOrEqual, err := le(arg1, arg2)
if err != nil {
return false, err
}
return !lessOrEqual, nil
}
// ge evaluates the comparison a >= b.
func ge(arg1, arg2 reflect.Value) (bool, error) {
// >= is the inverse of <.
lessThan, err := lt(arg1, arg2)
if err != nil {
return false, err
}
return !lessThan, nil
}
// HTML escaping.
var (
htmlQuot = []byte("&#34;") // shorter than "&quot;"
htmlApos = []byte("&#39;") // shorter than "&apos;" and apos was not in HTML until HTML5
htmlAmp = []byte("&amp;")
htmlLt = []byte("&lt;")
htmlGt = []byte("&gt;")
htmlNull = []byte("\uFFFD")
)
// HTMLEscape writes to w the escaped HTML equivalent of the plain text data b.
func HTMLEscape(w io.Writer, b []byte) {
last := 0
for i, c := range b {
var html []byte
switch c {
case '\000':
html = htmlNull
case '"':
html = htmlQuot
case '\'':
html = htmlApos
case '&':
html = htmlAmp
case '<':
html = htmlLt
case '>':
html = htmlGt
default:
continue
}
w.Write(b[last:i])
w.Write(html)
last = i + 1
}
w.Write(b[last:])
}
// HTMLEscapeString returns the escaped HTML equivalent of the plain text data s.
func HTMLEscapeString(s string) string {
// Avoid allocation if we can.
if !strings.ContainsAny(s, "'\"&<>\000") {
return s
}
var b bytes.Buffer
HTMLEscape(&b, []byte(s))
return b.String()
}
// HTMLEscaper returns the escaped HTML equivalent of the textual
// representation of its arguments.
func HTMLEscaper(args ...interface{}) string {
return HTMLEscapeString(evalArgs(args))
}
// JavaScript escaping.
var (
jsLowUni = []byte(`\u00`)
hex = []byte("0123456789ABCDEF")
jsBackslash = []byte(`\\`)
jsApos = []byte(`\'`)
jsQuot = []byte(`\"`)
jsLt = []byte(`\x3C`)
jsGt = []byte(`\x3E`)
)
// JSEscape writes to w the escaped JavaScript equivalent of the plain text data b.
func JSEscape(w io.Writer, b []byte) {
last := 0
for i := 0; i < len(b); i++ {
c := b[i]
if !jsIsSpecial(rune(c)) {
// fast path: nothing to do
continue
}
w.Write(b[last:i])
if c < utf8.RuneSelf {
// Quotes, slashes and angle brackets get quoted.
// Control characters get written as \u00XX.
switch c {
case '\\':
w.Write(jsBackslash)
case '\'':
w.Write(jsApos)
case '"':
w.Write(jsQuot)
case '<':
w.Write(jsLt)
case '>':
w.Write(jsGt)
default:
w.Write(jsLowUni)
t, b := c>>4, c&0x0f
w.Write(hex[t : t+1])
w.Write(hex[b : b+1])
}
} else {
// Unicode rune.
r, size := utf8.DecodeRune(b[i:])
if unicode.IsPrint(r) {
w.Write(b[i : i+size])
} else {
fmt.Fprintf(w, "\\u%04X", r)
}
i += size - 1
}
last = i + 1
}
w.Write(b[last:])
}
// JSEscapeString returns the escaped JavaScript equivalent of the plain text data s.
func JSEscapeString(s string) string {
// Avoid allocation if we can.
if strings.IndexFunc(s, jsIsSpecial) < 0 {
return s
}
var b bytes.Buffer
JSEscape(&b, []byte(s))
return b.String()
}
func jsIsSpecial(r rune) bool {
switch r {
case '\\', '\'', '"', '<', '>':
return true
}
return r < ' ' || utf8.RuneSelf <= r
}
// JSEscaper returns the escaped JavaScript equivalent of the textual
// representation of its arguments.
func JSEscaper(args ...interface{}) string {
return JSEscapeString(evalArgs(args))
}
// URLQueryEscaper returns the escaped value of the textual representation of
// its arguments in a form suitable for embedding in a URL query.
func URLQueryEscaper(args ...interface{}) string {
return url.QueryEscape(evalArgs(args))
}
// evalArgs formats the list of arguments into a string. It is therefore equivalent to
// fmt.Sprint(args...)
// except that each argument is indirected (if a pointer), as required,
// using the same rules as the default string evaluation during template
// execution.
func evalArgs(args []interface{}) string {
ok := false
var s string
// Fast path for simple common case.
if len(args) == 1 {
s, ok = args[0].(string)
}
if !ok {
for i, arg := range args {
a, ok := printableValue(reflect.ValueOf(arg))
if ok {
args[i] = a
} // else let fmt do its thing
}
s = fmt.Sprint(args...)
}
return s
}

View File

@ -0,0 +1,129 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Helper functions to make constructing templates easier.
package template
import (
"fmt"
"io/ioutil"
"path/filepath"
)
// Functions and methods to parse templates.
// Must is a helper that wraps a call to a function returning (*Template, error)
// and panics if the error is non-nil. It is intended for use in variable
// initializations such as
// var t = template.Must(template.New("name").Parse("text"))
func Must(t *Template, err error) *Template {
if err != nil {
panic(err)
}
return t
}
// ParseFiles creates a new Template and parses the template definitions from
// the named files. The returned template's name will have the base name and
// parsed contents of the first file. There must be at least one file.
// If an error occurs, parsing stops and the returned *Template is nil.
//
// When parsing multiple files with the same name in different directories,
// the last one mentioned will be the one that results.
// For instance, ParseFiles("a/foo", "b/foo") stores "b/foo" as the template
// named "foo", while "a/foo" is unavailable.
func ParseFiles(filenames ...string) (*Template, error) {
return parseFiles(nil, filenames...)
}
// ParseFiles parses the named files and associates the resulting templates with
// t. If an error occurs, parsing stops and the returned template is nil;
// otherwise it is t. There must be at least one file.
// Since the templates created by ParseFiles are named by the base
// names of the argument files, t should usually have the name of one
// of the (base) names of the files. If it does not, depending on t's
// contents before calling ParseFiles, t.Execute may fail. In that
// case use t.ExecuteTemplate to execute a valid template.
//
// When parsing multiple files with the same name in different directories,
// the last one mentioned will be the one that results.
func (t *Template) ParseFiles(filenames ...string) (*Template, error) {
t.init()
return parseFiles(t, filenames...)
}
// parseFiles is the helper for the method and function. If the argument
// template is nil, it is created from the first file.
func parseFiles(t *Template, filenames ...string) (*Template, error) {
if len(filenames) == 0 {
// Not really a problem, but be consistent.
return nil, fmt.Errorf("template: no files named in call to ParseFiles")
}
for _, filename := range filenames {
b, err := ioutil.ReadFile(filename)
if err != nil {
return nil, err
}
s := string(b)
name := filepath.Base(filename)
// First template becomes return value if not already defined,
// and we use that one for subsequent New calls to associate
// all the templates together. Also, if this file has the same name
// as t, this file becomes the contents of t, so
// t, err := New(name).Funcs(xxx).ParseFiles(name)
// works. Otherwise we create a new template associated with t.
var tmpl *Template
if t == nil {
t = New(name)
}
if name == t.Name() {
tmpl = t
} else {
tmpl = t.New(name)
}
_, err = tmpl.Parse(s)
if err != nil {
return nil, err
}
}
return t, nil
}
// ParseGlob creates a new Template and parses the template definitions from the
// files identified by the pattern, which must match at least one file. The
// returned template will have the (base) name and (parsed) contents of the
// first file matched by the pattern. ParseGlob is equivalent to calling
// ParseFiles with the list of files matched by the pattern.
//
// When parsing multiple files with the same name in different directories,
// the last one mentioned will be the one that results.
func ParseGlob(pattern string) (*Template, error) {
return parseGlob(nil, pattern)
}
// ParseGlob parses the template definitions in the files identified by the
// pattern and associates the resulting templates with t. The pattern is
// processed by filepath.Glob and must match at least one file. ParseGlob is
// equivalent to calling t.ParseFiles with the list of files matched by the
// pattern.
//
// When parsing multiple files with the same name in different directories,
// the last one mentioned will be the one that results.
func (t *Template) ParseGlob(pattern string) (*Template, error) {
t.init()
return parseGlob(t, pattern)
}
// parseGlob is the implementation of the function and method ParseGlob.
func parseGlob(t *Template, pattern string) (*Template, error) {
filenames, err := filepath.Glob(pattern)
if err != nil {
return nil, err
}
if len(filenames) == 0 {
return nil, fmt.Errorf("template: pattern matches no files: %#q", pattern)
}
return parseFiles(t, filenames...)
}

View File

@ -0,0 +1,74 @@
// Copyright 2015 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// This file contains the code to handle template options.
package template
import "strings"
// missingKeyAction defines how to respond to indexing a map with a key that is not present.
type missingKeyAction int
const (
mapInvalid missingKeyAction = iota // Return an invalid reflect.Value.
mapZeroValue // Return the zero value for the map element.
mapError // Error out
)
type option struct {
missingKey missingKeyAction
}
// Option sets options for the template. Options are described by
// strings, either a simple string or "key=value". There can be at
// most one equals sign in an option string. If the option string
// is unrecognized or otherwise invalid, Option panics.
//
// Known options:
//
// missingkey: Control the behavior during execution if a map is
// indexed with a key that is not present in the map.
// "missingkey=default" or "missingkey=invalid"
// The default behavior: Do nothing and continue execution.
// If printed, the result of the index operation is the string
// "<no value>".
// "missingkey=zero"
// The operation returns the zero value for the map type's element.
// "missingkey=error"
// Execution stops immediately with an error.
//
func (t *Template) Option(opt ...string) *Template {
t.init()
for _, s := range opt {
t.setOption(s)
}
return t
}
func (t *Template) setOption(opt string) {
if opt == "" {
panic("empty option string")
}
elems := strings.Split(opt, "=")
switch len(elems) {
case 2:
// key=value
switch elems[0] {
case "missingkey":
switch elems[1] {
case "invalid", "default":
t.option.missingKey = mapInvalid
return
case "zero":
t.option.missingKey = mapZeroValue
return
case "error":
t.option.missingKey = mapError
return
}
}
}
panic("unrecognized option: " + opt)
}

View File

@ -0,0 +1,642 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package parse
import (
"fmt"
"strings"
"unicode"
"unicode/utf8"
)
// item represents a token or text string returned from the scanner.
type item struct {
typ itemType // The type of this item.
pos Pos // The starting position, in bytes, of this item in the input string.
val string // The value of this item.
line int // The line number at the start of this item.
}
func (i item) String() string {
switch {
case i.typ == itemEOF:
return "EOF"
case i.typ == itemError:
return i.val
case i.typ > itemKeyword:
return fmt.Sprintf("<%s>", i.val)
case len(i.val) > 10:
return fmt.Sprintf("%.10q...", i.val)
}
return fmt.Sprintf("%q", i.val)
}
// itemType identifies the type of lex items.
type itemType int
const (
itemError itemType = iota // error occurred; value is text of error
itemBool // boolean constant
itemChar // printable ASCII character; grab bag for comma etc.
itemCharConstant // character constant
itemComplex // complex constant (1+2i); imaginary is just a number
itemAssign // equals ('=') introducing an assignment
itemDeclare // colon-equals (':=') introducing a declaration
itemEOF
itemField // alphanumeric identifier starting with '.'
itemIdentifier // alphanumeric identifier not starting with '.'
itemLeftDelim // left action delimiter
itemLeftParen // '(' inside action
itemNumber // simple number, including imaginary
itemPipe // pipe symbol
itemRawString // raw quoted string (includes quotes)
itemRightDelim // right action delimiter
itemRightParen // ')' inside action
itemSpace // run of spaces separating arguments
itemString // quoted string (includes quotes)
itemText // plain text
itemVariable // variable starting with '$', such as '$' or '$1' or '$hello'
// Keywords appear after all the rest.
itemKeyword // used only to delimit the keywords
itemBlock // block keyword
itemDot // the cursor, spelled '.'
itemDefine // define keyword
itemElse // else keyword
itemEnd // end keyword
itemIf // if keyword
itemNil // the untyped nil constant, easiest to treat as a keyword
itemRange // range keyword
itemTemplate // template keyword
itemWith // with keyword
)
var key = map[string]itemType{
".": itemDot,
"block": itemBlock,
"define": itemDefine,
"else": itemElse,
"end": itemEnd,
"if": itemIf,
"range": itemRange,
"nil": itemNil,
"template": itemTemplate,
"with": itemWith,
}
const eof = -1
// Trimming spaces.
// If the action begins "{{- " rather than "{{", then all space/tab/newlines
// preceding the action are trimmed; conversely if it ends " -}}" the
// leading spaces are trimmed. This is done entirely in the lexer; the
// parser never sees it happen. We require an ASCII space to be
// present to avoid ambiguity with things like "{{-3}}". It reads
// better with the space present anyway. For simplicity, only ASCII
// space does the job.
const (
spaceChars = " \t\r\n" // These are the space characters defined by Go itself.
leftTrimMarker = "- " // Attached to left delimiter, trims trailing spaces from preceding text.
rightTrimMarker = " -" // Attached to right delimiter, trims leading spaces from following text.
trimMarkerLen = Pos(len(leftTrimMarker))
)
// stateFn represents the state of the scanner as a function that returns the next state.
type stateFn func(*lexer) stateFn
// lexer holds the state of the scanner.
type lexer struct {
name string // the name of the input; used only for error reports
input string // the string being scanned
leftDelim string // start of action
rightDelim string // end of action
pos Pos // current position in the input
start Pos // start position of this item
width Pos // width of last rune read from input
items chan item // channel of scanned items
parenDepth int // nesting depth of ( ) exprs
line int // 1+number of newlines seen
}
// next returns the next rune in the input.
func (l *lexer) next() rune {
if int(l.pos) >= len(l.input) {
l.width = 0
return eof
}
r, w := utf8.DecodeRuneInString(l.input[l.pos:])
l.width = Pos(w)
l.pos += l.width
if r == '\n' {
l.line++
}
return r
}
// peek returns but does not consume the next rune in the input.
func (l *lexer) peek() rune {
r := l.next()
l.backup()
return r
}
// backup steps back one rune. Can only be called once per call of next.
func (l *lexer) backup() {
l.pos -= l.width
// Correct newline count.
if l.width == 1 && l.input[l.pos] == '\n' {
l.line--
}
}
// emit passes an item back to the client.
func (l *lexer) emit(t itemType) {
l.items <- item{t, l.start, l.input[l.start:l.pos], l.line}
// Some items contain text internally. If so, count their newlines.
switch t {
case itemText, itemRawString, itemLeftDelim, itemRightDelim:
l.line += strings.Count(l.input[l.start:l.pos], "\n")
}
l.start = l.pos
}
// ignore skips over the pending input before this point.
func (l *lexer) ignore() {
l.line += strings.Count(l.input[l.start:l.pos], "\n")
l.start = l.pos
}
// accept consumes the next rune if it's from the valid set.
func (l *lexer) accept(valid string) bool {
if strings.ContainsRune(valid, l.next()) {
return true
}
l.backup()
return false
}
// acceptRun consumes a run of runes from the valid set.
func (l *lexer) acceptRun(valid string) {
for strings.ContainsRune(valid, l.next()) {
}
l.backup()
}
// errorf returns an error token and terminates the scan by passing
// back a nil pointer that will be the next state, terminating l.nextItem.
func (l *lexer) errorf(format string, args ...interface{}) stateFn {
l.items <- item{itemError, l.start, fmt.Sprintf(format, args...), l.line}
return nil
}
// nextItem returns the next item from the input.
// Called by the parser, not in the lexing goroutine.
func (l *lexer) nextItem() item {
return <-l.items
}
// drain drains the output so the lexing goroutine will exit.
// Called by the parser, not in the lexing goroutine.
func (l *lexer) drain() {
for range l.items {
}
}
// lex creates a new scanner for the input string.
func lex(name, input, left, right string) *lexer {
if left == "" {
left = leftDelim
}
if right == "" {
right = rightDelim
}
l := &lexer{
name: name,
input: input,
leftDelim: left,
rightDelim: right,
items: make(chan item),
line: 1,
}
go l.run()
return l
}
// run runs the state machine for the lexer.
func (l *lexer) run() {
for state := lexText; state != nil; {
state = state(l)
}
close(l.items)
}
// state functions
const (
leftDelim = "{{"
rightDelim = "}}"
leftComment = "/*"
rightComment = "*/"
)
// lexText scans until an opening action delimiter, "{{".
func lexText(l *lexer) stateFn {
l.width = 0
if x := strings.Index(l.input[l.pos:], l.leftDelim); x >= 0 {
ldn := Pos(len(l.leftDelim))
l.pos += Pos(x)
trimLength := Pos(0)
if strings.HasPrefix(l.input[l.pos+ldn:], leftTrimMarker) {
trimLength = rightTrimLength(l.input[l.start:l.pos])
}
l.pos -= trimLength
if l.pos > l.start {
l.emit(itemText)
}
l.pos += trimLength
l.ignore()
return lexLeftDelim
} else {
l.pos = Pos(len(l.input))
}
// Correctly reached EOF.
if l.pos > l.start {
l.emit(itemText)
}
l.emit(itemEOF)
return nil
}
// rightTrimLength returns the length of the spaces at the end of the string.
func rightTrimLength(s string) Pos {
return Pos(len(s) - len(strings.TrimRight(s, spaceChars)))
}
// atRightDelim reports whether the lexer is at a right delimiter, possibly preceded by a trim marker.
func (l *lexer) atRightDelim() (delim, trimSpaces bool) {
if strings.HasPrefix(l.input[l.pos:], l.rightDelim) {
return true, false
}
// The right delim might have the marker before.
if strings.HasPrefix(l.input[l.pos:], rightTrimMarker) &&
strings.HasPrefix(l.input[l.pos+trimMarkerLen:], l.rightDelim) {
return true, true
}
return false, false
}
// leftTrimLength returns the length of the spaces at the beginning of the string.
func leftTrimLength(s string) Pos {
return Pos(len(s) - len(strings.TrimLeft(s, spaceChars)))
}
// lexLeftDelim scans the left delimiter, which is known to be present, possibly with a trim marker.
func lexLeftDelim(l *lexer) stateFn {
l.pos += Pos(len(l.leftDelim))
trimSpace := strings.HasPrefix(l.input[l.pos:], leftTrimMarker)
afterMarker := Pos(0)
if trimSpace {
afterMarker = trimMarkerLen
}
if strings.HasPrefix(l.input[l.pos+afterMarker:], leftComment) {
l.pos += afterMarker
l.ignore()
return lexComment
}
l.emit(itemLeftDelim)
l.pos += afterMarker
l.ignore()
l.parenDepth = 0
return lexInsideAction
}
// lexComment scans a comment. The left comment marker is known to be present.
func lexComment(l *lexer) stateFn {
l.pos += Pos(len(leftComment))
i := strings.Index(l.input[l.pos:], rightComment)
if i < 0 {
return l.errorf("unclosed comment")
}
l.pos += Pos(i + len(rightComment))
delim, trimSpace := l.atRightDelim()
if !delim {
return l.errorf("comment ends before closing delimiter")
}
if trimSpace {
l.pos += trimMarkerLen
}
l.pos += Pos(len(l.rightDelim))
if trimSpace {
l.pos += leftTrimLength(l.input[l.pos:])
}
l.ignore()
return lexText
}
// lexRightDelim scans the right delimiter, which is known to be present, possibly with a trim marker.
func lexRightDelim(l *lexer) stateFn {
trimSpace := strings.HasPrefix(l.input[l.pos:], rightTrimMarker)
if trimSpace {
l.pos += trimMarkerLen
l.ignore()
}
l.pos += Pos(len(l.rightDelim))
l.emit(itemRightDelim)
if trimSpace {
l.pos += leftTrimLength(l.input[l.pos:])
l.ignore()
}
return lexText
}
// lexInsideAction scans the elements inside action delimiters.
func lexInsideAction(l *lexer) stateFn {
// Either number, quoted string, or identifier.
// Spaces separate arguments; runs of spaces turn into itemSpace.
// Pipe symbols separate and are emitted.
delim, _ := l.atRightDelim()
if delim {
if l.parenDepth == 0 {
return lexRightDelim
}
return l.errorf("unclosed left paren")
}
switch r := l.next(); {
case r == eof || isEndOfLine(r):
return l.errorf("unclosed action")
case isSpace(r):
return lexSpace
case r == '=':
l.emit(itemAssign)
case r == ':':
if l.next() != '=' {
return l.errorf("expected :=")
}
l.emit(itemDeclare)
case r == '|':
l.emit(itemPipe)
case r == '"':
return lexQuote
case r == '`':
return lexRawQuote
case r == '$':
return lexVariable
case r == '\'':
return lexChar
case r == '.':
// special look-ahead for ".field" so we don't break l.backup().
if l.pos < Pos(len(l.input)) {
r := l.input[l.pos]
if r < '0' || '9' < r {
return lexField
}
}
fallthrough // '.' can start a number.
case r == '+' || r == '-' || ('0' <= r && r <= '9'):
l.backup()
return lexNumber
case isAlphaNumeric(r):
l.backup()
return lexIdentifier
case r == '(':
l.emit(itemLeftParen)
l.parenDepth++
case r == ')':
l.emit(itemRightParen)
l.parenDepth--
if l.parenDepth < 0 {
return l.errorf("unexpected right paren %#U", r)
}
case r <= unicode.MaxASCII && unicode.IsPrint(r):
l.emit(itemChar)
return lexInsideAction
default:
return l.errorf("unrecognized character in action: %#U", r)
}
return lexInsideAction
}
// lexSpace scans a run of space characters.
// One space has already been seen.
func lexSpace(l *lexer) stateFn {
for isSpace(l.peek()) {
l.next()
}
l.emit(itemSpace)
return lexInsideAction
}
// lexIdentifier scans an alphanumeric.
func lexIdentifier(l *lexer) stateFn {
Loop:
for {
switch r := l.next(); {
case isAlphaNumeric(r):
// absorb.
default:
l.backup()
word := l.input[l.start:l.pos]
if !l.atTerminator() {
return l.errorf("bad character %#U", r)
}
switch {
case key[word] > itemKeyword:
l.emit(key[word])
case word[0] == '.':
l.emit(itemField)
case word == "true", word == "false":
l.emit(itemBool)
default:
l.emit(itemIdentifier)
}
break Loop
}
}
return lexInsideAction
}
// lexField scans a field: .Alphanumeric.
// The . has been scanned.
func lexField(l *lexer) stateFn {
return lexFieldOrVariable(l, itemField)
}
// lexVariable scans a Variable: $Alphanumeric.
// The $ has been scanned.
func lexVariable(l *lexer) stateFn {
if l.atTerminator() { // Nothing interesting follows -> "$".
l.emit(itemVariable)
return lexInsideAction
}
return lexFieldOrVariable(l, itemVariable)
}
// lexVariable scans a field or variable: [.$]Alphanumeric.
// The . or $ has been scanned.
func lexFieldOrVariable(l *lexer, typ itemType) stateFn {
if l.atTerminator() { // Nothing interesting follows -> "." or "$".
if typ == itemVariable {
l.emit(itemVariable)
} else {
l.emit(itemDot)
}
return lexInsideAction
}
var r rune
for {
r = l.next()
if !isAlphaNumeric(r) {
l.backup()
break
}
}
if !l.atTerminator() {
return l.errorf("bad character %#U", r)
}
l.emit(typ)
return lexInsideAction
}
// atTerminator reports whether the input is at valid termination character to
// appear after an identifier. Breaks .X.Y into two pieces. Also catches cases
// like "$x+2" not being acceptable without a space, in case we decide one
// day to implement arithmetic.
func (l *lexer) atTerminator() bool {
r := l.peek()
if isSpace(r) || isEndOfLine(r) {
return true
}
switch r {
case eof, '.', ',', '|', ':', ')', '(':
return true
}
// Does r start the delimiter? This can be ambiguous (with delim=="//", $x/2 will
// succeed but should fail) but only in extremely rare cases caused by willfully
// bad choice of delimiter.
if rd, _ := utf8.DecodeRuneInString(l.rightDelim); rd == r {
return true
}
return false
}
// lexChar scans a character constant. The initial quote is already
// scanned. Syntax checking is done by the parser.
func lexChar(l *lexer) stateFn {
Loop:
for {
switch l.next() {
case '\\':
if r := l.next(); r != eof && r != '\n' {
break
}
fallthrough
case eof, '\n':
return l.errorf("unterminated character constant")
case '\'':
break Loop
}
}
l.emit(itemCharConstant)
return lexInsideAction
}
// lexNumber scans a number: decimal, octal, hex, float, or imaginary. This
// isn't a perfect number scanner - for instance it accepts "." and "0x0.2"
// and "089" - but when it's wrong the input is invalid and the parser (via
// strconv) will notice.
func lexNumber(l *lexer) stateFn {
if !l.scanNumber() {
return l.errorf("bad number syntax: %q", l.input[l.start:l.pos])
}
if sign := l.peek(); sign == '+' || sign == '-' {
// Complex: 1+2i. No spaces, must end in 'i'.
if !l.scanNumber() || l.input[l.pos-1] != 'i' {
return l.errorf("bad number syntax: %q", l.input[l.start:l.pos])
}
l.emit(itemComplex)
} else {
l.emit(itemNumber)
}
return lexInsideAction
}
func (l *lexer) scanNumber() bool {
// Optional leading sign.
l.accept("+-")
// Is it hex?
digits := "0123456789"
if l.accept("0") && l.accept("xX") {
digits = "0123456789abcdefABCDEF"
}
l.acceptRun(digits)
if l.accept(".") {
l.acceptRun(digits)
}
if l.accept("eE") {
l.accept("+-")
l.acceptRun("0123456789")
}
// Is it imaginary?
l.accept("i")
// Next thing mustn't be alphanumeric.
if isAlphaNumeric(l.peek()) {
l.next()
return false
}
return true
}
// lexQuote scans a quoted string.
func lexQuote(l *lexer) stateFn {
Loop:
for {
switch l.next() {
case '\\':
if r := l.next(); r != eof && r != '\n' {
break
}
fallthrough
case eof, '\n':
return l.errorf("unterminated quoted string")
case '"':
break Loop
}
}
l.emit(itemString)
return lexInsideAction
}
// lexRawQuote scans a raw quoted string.
func lexRawQuote(l *lexer) stateFn {
startLine := l.line
Loop:
for {
switch l.next() {
case eof:
// Restore line number to location of opening quote.
// We will error out so it's ok just to overwrite the field.
l.line = startLine
return l.errorf("unterminated raw quoted string")
case '`':
break Loop
}
}
l.emit(itemRawString)
return lexInsideAction
}
// isSpace reports whether r is a space character.
func isSpace(r rune) bool {
return r == ' ' || r == '\t'
}
// isEndOfLine reports whether r is an end-of-line character.
func isEndOfLine(r rune) bool {
return r == '\r' || r == '\n'
}
// isAlphaNumeric reports whether r is an alphabetic, digit, or underscore.
func isAlphaNumeric(r rune) bool {
return r == '_' || unicode.IsLetter(r) || unicode.IsDigit(r)
}

View File

@ -0,0 +1,841 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Parse nodes.
package parse
import (
"bytes"
"fmt"
"strconv"
"strings"
)
var textFormat = "%s" // Changed to "%q" in tests for better error messages.
// A Node is an element in the parse tree. The interface is trivial.
// The interface contains an unexported method so that only
// types local to this package can satisfy it.
type Node interface {
Type() NodeType
String() string
// Copy does a deep copy of the Node and all its components.
// To avoid type assertions, some XxxNodes also have specialized
// CopyXxx methods that return *XxxNode.
Copy() Node
Position() Pos // byte position of start of node in full original input string
// tree returns the containing *Tree.
// It is unexported so all implementations of Node are in this package.
tree() *Tree
}
// NodeType identifies the type of a parse tree node.
type NodeType int
// Pos represents a byte position in the original input text from which
// this template was parsed.
type Pos int
func (p Pos) Position() Pos {
return p
}
// Type returns itself and provides an easy default implementation
// for embedding in a Node. Embedded in all non-trivial Nodes.
func (t NodeType) Type() NodeType {
return t
}
const (
NodeText NodeType = iota // Plain text.
NodeAction // A non-control action such as a field evaluation.
NodeBool // A boolean constant.
NodeChain // A sequence of field accesses.
NodeCommand // An element of a pipeline.
NodeDot // The cursor, dot.
nodeElse // An else action. Not added to tree.
nodeEnd // An end action. Not added to tree.
NodeField // A field or method name.
NodeIdentifier // An identifier; always a function name.
NodeIf // An if action.
NodeList // A list of Nodes.
NodeNil // An untyped nil constant.
NodeNumber // A numerical constant.
NodePipe // A pipeline of commands.
NodeRange // A range action.
NodeString // A string constant.
NodeTemplate // A template invocation action.
NodeVariable // A $ variable.
NodeWith // A with action.
)
// Nodes.
// ListNode holds a sequence of nodes.
type ListNode struct {
NodeType
Pos
tr *Tree
Nodes []Node // The element nodes in lexical order.
}
func (t *Tree) newList(pos Pos) *ListNode {
return &ListNode{tr: t, NodeType: NodeList, Pos: pos}
}
func (l *ListNode) append(n Node) {
l.Nodes = append(l.Nodes, n)
}
func (l *ListNode) tree() *Tree {
return l.tr
}
func (l *ListNode) String() string {
b := new(bytes.Buffer)
for _, n := range l.Nodes {
fmt.Fprint(b, n)
}
return b.String()
}
func (l *ListNode) CopyList() *ListNode {
if l == nil {
return l
}
n := l.tr.newList(l.Pos)
for _, elem := range l.Nodes {
n.append(elem.Copy())
}
return n
}
func (l *ListNode) Copy() Node {
return l.CopyList()
}
// TextNode holds plain text.
type TextNode struct {
NodeType
Pos
tr *Tree
Text []byte // The text; may span newlines.
}
func (t *Tree) newText(pos Pos, text string) *TextNode {
return &TextNode{tr: t, NodeType: NodeText, Pos: pos, Text: []byte(text)}
}
func (t *TextNode) String() string {
return fmt.Sprintf(textFormat, t.Text)
}
func (t *TextNode) tree() *Tree {
return t.tr
}
func (t *TextNode) Copy() Node {
return &TextNode{tr: t.tr, NodeType: NodeText, Pos: t.Pos, Text: append([]byte{}, t.Text...)}
}
// PipeNode holds a pipeline with optional declaration
type PipeNode struct {
NodeType
Pos
tr *Tree
Line int // The line number in the input. Deprecated: Kept for compatibility.
IsAssign bool // The variables are being assigned, not declared.
Decl []*VariableNode // Variables in lexical order.
Cmds []*CommandNode // The commands in lexical order.
}
func (t *Tree) newPipeline(pos Pos, line int, vars []*VariableNode) *PipeNode {
return &PipeNode{tr: t, NodeType: NodePipe, Pos: pos, Line: line, Decl: vars}
}
func (p *PipeNode) append(command *CommandNode) {
p.Cmds = append(p.Cmds, command)
}
func (p *PipeNode) String() string {
s := ""
if len(p.Decl) > 0 {
for i, v := range p.Decl {
if i > 0 {
s += ", "
}
s += v.String()
}
s += " := "
}
for i, c := range p.Cmds {
if i > 0 {
s += " | "
}
s += c.String()
}
return s
}
func (p *PipeNode) tree() *Tree {
return p.tr
}
func (p *PipeNode) CopyPipe() *PipeNode {
if p == nil {
return p
}
var vars []*VariableNode
for _, d := range p.Decl {
vars = append(vars, d.Copy().(*VariableNode))
}
n := p.tr.newPipeline(p.Pos, p.Line, vars)
n.IsAssign = p.IsAssign
for _, c := range p.Cmds {
n.append(c.Copy().(*CommandNode))
}
return n
}
func (p *PipeNode) Copy() Node {
return p.CopyPipe()
}
// ActionNode holds an action (something bounded by delimiters).
// Control actions have their own nodes; ActionNode represents simple
// ones such as field evaluations and parenthesized pipelines.
type ActionNode struct {
NodeType
Pos
tr *Tree
Line int // The line number in the input. Deprecated: Kept for compatibility.
Pipe *PipeNode // The pipeline in the action.
}
func (t *Tree) newAction(pos Pos, line int, pipe *PipeNode) *ActionNode {
return &ActionNode{tr: t, NodeType: NodeAction, Pos: pos, Line: line, Pipe: pipe}
}
func (a *ActionNode) String() string {
return fmt.Sprintf("{{%s}}", a.Pipe)
}
func (a *ActionNode) tree() *Tree {
return a.tr
}
func (a *ActionNode) Copy() Node {
return a.tr.newAction(a.Pos, a.Line, a.Pipe.CopyPipe())
}
// CommandNode holds a command (a pipeline inside an evaluating action).
type CommandNode struct {
NodeType
Pos
tr *Tree
Args []Node // Arguments in lexical order: Identifier, field, or constant.
}
func (t *Tree) newCommand(pos Pos) *CommandNode {
return &CommandNode{tr: t, NodeType: NodeCommand, Pos: pos}
}
func (c *CommandNode) append(arg Node) {
c.Args = append(c.Args, arg)
}
func (c *CommandNode) String() string {
s := ""
for i, arg := range c.Args {
if i > 0 {
s += " "
}
if arg, ok := arg.(*PipeNode); ok {
s += "(" + arg.String() + ")"
continue
}
s += arg.String()
}
return s
}
func (c *CommandNode) tree() *Tree {
return c.tr
}
func (c *CommandNode) Copy() Node {
if c == nil {
return c
}
n := c.tr.newCommand(c.Pos)
for _, c := range c.Args {
n.append(c.Copy())
}
return n
}
// IdentifierNode holds an identifier.
type IdentifierNode struct {
NodeType
Pos
tr *Tree
Ident string // The identifier's name.
}
// NewIdentifier returns a new IdentifierNode with the given identifier name.
func NewIdentifier(ident string) *IdentifierNode {
return &IdentifierNode{NodeType: NodeIdentifier, Ident: ident}
}
// SetPos sets the position. NewIdentifier is a public method so we can't modify its signature.
// Chained for convenience.
// TODO: fix one day?
func (i *IdentifierNode) SetPos(pos Pos) *IdentifierNode {
i.Pos = pos
return i
}
// SetTree sets the parent tree for the node. NewIdentifier is a public method so we can't modify its signature.
// Chained for convenience.
// TODO: fix one day?
func (i *IdentifierNode) SetTree(t *Tree) *IdentifierNode {
i.tr = t
return i
}
func (i *IdentifierNode) String() string {
return i.Ident
}
func (i *IdentifierNode) tree() *Tree {
return i.tr
}
func (i *IdentifierNode) Copy() Node {
return NewIdentifier(i.Ident).SetTree(i.tr).SetPos(i.Pos)
}
// AssignNode holds a list of variable names, possibly with chained field
// accesses. The dollar sign is part of the (first) name.
type VariableNode struct {
NodeType
Pos
tr *Tree
Ident []string // Variable name and fields in lexical order.
}
func (t *Tree) newVariable(pos Pos, ident string) *VariableNode {
return &VariableNode{tr: t, NodeType: NodeVariable, Pos: pos, Ident: strings.Split(ident, ".")}
}
func (v *VariableNode) String() string {
s := ""
for i, id := range v.Ident {
if i > 0 {
s += "."
}
s += id
}
return s
}
func (v *VariableNode) tree() *Tree {
return v.tr
}
func (v *VariableNode) Copy() Node {
return &VariableNode{tr: v.tr, NodeType: NodeVariable, Pos: v.Pos, Ident: append([]string{}, v.Ident...)}
}
// DotNode holds the special identifier '.'.
type DotNode struct {
NodeType
Pos
tr *Tree
}
func (t *Tree) newDot(pos Pos) *DotNode {
return &DotNode{tr: t, NodeType: NodeDot, Pos: pos}
}
func (d *DotNode) Type() NodeType {
// Override method on embedded NodeType for API compatibility.
// TODO: Not really a problem; could change API without effect but
// api tool complains.
return NodeDot
}
func (d *DotNode) String() string {
return "."
}
func (d *DotNode) tree() *Tree {
return d.tr
}
func (d *DotNode) Copy() Node {
return d.tr.newDot(d.Pos)
}
// NilNode holds the special identifier 'nil' representing an untyped nil constant.
type NilNode struct {
NodeType
Pos
tr *Tree
}
func (t *Tree) newNil(pos Pos) *NilNode {
return &NilNode{tr: t, NodeType: NodeNil, Pos: pos}
}
func (n *NilNode) Type() NodeType {
// Override method on embedded NodeType for API compatibility.
// TODO: Not really a problem; could change API without effect but
// api tool complains.
return NodeNil
}
func (n *NilNode) String() string {
return "nil"
}
func (n *NilNode) tree() *Tree {
return n.tr
}
func (n *NilNode) Copy() Node {
return n.tr.newNil(n.Pos)
}
// FieldNode holds a field (identifier starting with '.').
// The names may be chained ('.x.y').
// The period is dropped from each ident.
type FieldNode struct {
NodeType
Pos
tr *Tree
Ident []string // The identifiers in lexical order.
}
func (t *Tree) newField(pos Pos, ident string) *FieldNode {
return &FieldNode{tr: t, NodeType: NodeField, Pos: pos, Ident: strings.Split(ident[1:], ".")} // [1:] to drop leading period
}
func (f *FieldNode) String() string {
s := ""
for _, id := range f.Ident {
s += "." + id
}
return s
}
func (f *FieldNode) tree() *Tree {
return f.tr
}
func (f *FieldNode) Copy() Node {
return &FieldNode{tr: f.tr, NodeType: NodeField, Pos: f.Pos, Ident: append([]string{}, f.Ident...)}
}
// ChainNode holds a term followed by a chain of field accesses (identifier starting with '.').
// The names may be chained ('.x.y').
// The periods are dropped from each ident.
type ChainNode struct {
NodeType
Pos
tr *Tree
Node Node
Field []string // The identifiers in lexical order.
}
func (t *Tree) newChain(pos Pos, node Node) *ChainNode {
return &ChainNode{tr: t, NodeType: NodeChain, Pos: pos, Node: node}
}
// Add adds the named field (which should start with a period) to the end of the chain.
func (c *ChainNode) Add(field string) {
if len(field) == 0 || field[0] != '.' {
panic("no dot in field")
}
field = field[1:] // Remove leading dot.
if field == "" {
panic("empty field")
}
c.Field = append(c.Field, field)
}
func (c *ChainNode) String() string {
s := c.Node.String()
if _, ok := c.Node.(*PipeNode); ok {
s = "(" + s + ")"
}
for _, field := range c.Field {
s += "." + field
}
return s
}
func (c *ChainNode) tree() *Tree {
return c.tr
}
func (c *ChainNode) Copy() Node {
return &ChainNode{tr: c.tr, NodeType: NodeChain, Pos: c.Pos, Node: c.Node, Field: append([]string{}, c.Field...)}
}
// BoolNode holds a boolean constant.
type BoolNode struct {
NodeType
Pos
tr *Tree
True bool // The value of the boolean constant.
}
func (t *Tree) newBool(pos Pos, true bool) *BoolNode {
return &BoolNode{tr: t, NodeType: NodeBool, Pos: pos, True: true}
}
func (b *BoolNode) String() string {
if b.True {
return "true"
}
return "false"
}
func (b *BoolNode) tree() *Tree {
return b.tr
}
func (b *BoolNode) Copy() Node {
return b.tr.newBool(b.Pos, b.True)
}
// NumberNode holds a number: signed or unsigned integer, float, or complex.
// The value is parsed and stored under all the types that can represent the value.
// This simulates in a small amount of code the behavior of Go's ideal constants.
type NumberNode struct {
NodeType
Pos
tr *Tree
IsInt bool // Number has an integral value.
IsUint bool // Number has an unsigned integral value.
IsFloat bool // Number has a floating-point value.
IsComplex bool // Number is complex.
Int64 int64 // The signed integer value.
Uint64 uint64 // The unsigned integer value.
Float64 float64 // The floating-point value.
Complex128 complex128 // The complex value.
Text string // The original textual representation from the input.
}
func (t *Tree) newNumber(pos Pos, text string, typ itemType) (*NumberNode, error) {
n := &NumberNode{tr: t, NodeType: NodeNumber, Pos: pos, Text: text}
switch typ {
case itemCharConstant:
rune, _, tail, err := strconv.UnquoteChar(text[1:], text[0])
if err != nil {
return nil, err
}
if tail != "'" {
return nil, fmt.Errorf("malformed character constant: %s", text)
}
n.Int64 = int64(rune)
n.IsInt = true
n.Uint64 = uint64(rune)
n.IsUint = true
n.Float64 = float64(rune) // odd but those are the rules.
n.IsFloat = true
return n, nil
case itemComplex:
// fmt.Sscan can parse the pair, so let it do the work.
if _, err := fmt.Sscan(text, &n.Complex128); err != nil {
return nil, err
}
n.IsComplex = true
n.simplifyComplex()
return n, nil
}
// Imaginary constants can only be complex unless they are zero.
if len(text) > 0 && text[len(text)-1] == 'i' {
f, err := strconv.ParseFloat(text[:len(text)-1], 64)
if err == nil {
n.IsComplex = true
n.Complex128 = complex(0, f)
n.simplifyComplex()
return n, nil
}
}
// Do integer test first so we get 0x123 etc.
u, err := strconv.ParseUint(text, 0, 64) // will fail for -0; fixed below.
if err == nil {
n.IsUint = true
n.Uint64 = u
}
i, err := strconv.ParseInt(text, 0, 64)
if err == nil {
n.IsInt = true
n.Int64 = i
if i == 0 {
n.IsUint = true // in case of -0.
n.Uint64 = u
}
}
// If an integer extraction succeeded, promote the float.
if n.IsInt {
n.IsFloat = true
n.Float64 = float64(n.Int64)
} else if n.IsUint {
n.IsFloat = true
n.Float64 = float64(n.Uint64)
} else {
f, err := strconv.ParseFloat(text, 64)
if err == nil {
// If we parsed it as a float but it looks like an integer,
// it's a huge number too large to fit in an int. Reject it.
if !strings.ContainsAny(text, ".eE") {
return nil, fmt.Errorf("integer overflow: %q", text)
}
n.IsFloat = true
n.Float64 = f
// If a floating-point extraction succeeded, extract the int if needed.
if !n.IsInt && float64(int64(f)) == f {
n.IsInt = true
n.Int64 = int64(f)
}
if !n.IsUint && float64(uint64(f)) == f {
n.IsUint = true
n.Uint64 = uint64(f)
}
}
}
if !n.IsInt && !n.IsUint && !n.IsFloat {
return nil, fmt.Errorf("illegal number syntax: %q", text)
}
return n, nil
}
// simplifyComplex pulls out any other types that are represented by the complex number.
// These all require that the imaginary part be zero.
func (n *NumberNode) simplifyComplex() {
n.IsFloat = imag(n.Complex128) == 0
if n.IsFloat {
n.Float64 = real(n.Complex128)
n.IsInt = float64(int64(n.Float64)) == n.Float64
if n.IsInt {
n.Int64 = int64(n.Float64)
}
n.IsUint = float64(uint64(n.Float64)) == n.Float64
if n.IsUint {
n.Uint64 = uint64(n.Float64)
}
}
}
func (n *NumberNode) String() string {
return n.Text
}
func (n *NumberNode) tree() *Tree {
return n.tr
}
func (n *NumberNode) Copy() Node {
nn := new(NumberNode)
*nn = *n // Easy, fast, correct.
return nn
}
// StringNode holds a string constant. The value has been "unquoted".
type StringNode struct {
NodeType
Pos
tr *Tree
Quoted string // The original text of the string, with quotes.
Text string // The string, after quote processing.
}
func (t *Tree) newString(pos Pos, orig, text string) *StringNode {
return &StringNode{tr: t, NodeType: NodeString, Pos: pos, Quoted: orig, Text: text}
}
func (s *StringNode) String() string {
return s.Quoted
}
func (s *StringNode) tree() *Tree {
return s.tr
}
func (s *StringNode) Copy() Node {
return s.tr.newString(s.Pos, s.Quoted, s.Text)
}
// endNode represents an {{end}} action.
// It does not appear in the final parse tree.
type endNode struct {
NodeType
Pos
tr *Tree
}
func (t *Tree) newEnd(pos Pos) *endNode {
return &endNode{tr: t, NodeType: nodeEnd, Pos: pos}
}
func (e *endNode) String() string {
return "{{end}}"
}
func (e *endNode) tree() *Tree {
return e.tr
}
func (e *endNode) Copy() Node {
return e.tr.newEnd(e.Pos)
}
// elseNode represents an {{else}} action. Does not appear in the final tree.
type elseNode struct {
NodeType
Pos
tr *Tree
Line int // The line number in the input. Deprecated: Kept for compatibility.
}
func (t *Tree) newElse(pos Pos, line int) *elseNode {
return &elseNode{tr: t, NodeType: nodeElse, Pos: pos, Line: line}
}
func (e *elseNode) Type() NodeType {
return nodeElse
}
func (e *elseNode) String() string {
return "{{else}}"
}
func (e *elseNode) tree() *Tree {
return e.tr
}
func (e *elseNode) Copy() Node {
return e.tr.newElse(e.Pos, e.Line)
}
// BranchNode is the common representation of if, range, and with.
type BranchNode struct {
NodeType
Pos
tr *Tree
Line int // The line number in the input. Deprecated: Kept for compatibility.
Pipe *PipeNode // The pipeline to be evaluated.
List *ListNode // What to execute if the value is non-empty.
ElseList *ListNode // What to execute if the value is empty (nil if absent).
}
func (b *BranchNode) String() string {
name := ""
switch b.NodeType {
case NodeIf:
name = "if"
case NodeRange:
name = "range"
case NodeWith:
name = "with"
default:
panic("unknown branch type")
}
if b.ElseList != nil {
return fmt.Sprintf("{{%s %s}}%s{{else}}%s{{end}}", name, b.Pipe, b.List, b.ElseList)
}
return fmt.Sprintf("{{%s %s}}%s{{end}}", name, b.Pipe, b.List)
}
func (b *BranchNode) tree() *Tree {
return b.tr
}
func (b *BranchNode) Copy() Node {
switch b.NodeType {
case NodeIf:
return b.tr.newIf(b.Pos, b.Line, b.Pipe, b.List, b.ElseList)
case NodeRange:
return b.tr.newRange(b.Pos, b.Line, b.Pipe, b.List, b.ElseList)
case NodeWith:
return b.tr.newWith(b.Pos, b.Line, b.Pipe, b.List, b.ElseList)
default:
panic("unknown branch type")
}
}
// IfNode represents an {{if}} action and its commands.
type IfNode struct {
BranchNode
}
func (t *Tree) newIf(pos Pos, line int, pipe *PipeNode, list, elseList *ListNode) *IfNode {
return &IfNode{BranchNode{tr: t, NodeType: NodeIf, Pos: pos, Line: line, Pipe: pipe, List: list, ElseList: elseList}}
}
func (i *IfNode) Copy() Node {
return i.tr.newIf(i.Pos, i.Line, i.Pipe.CopyPipe(), i.List.CopyList(), i.ElseList.CopyList())
}
// RangeNode represents a {{range}} action and its commands.
type RangeNode struct {
BranchNode
}
func (t *Tree) newRange(pos Pos, line int, pipe *PipeNode, list, elseList *ListNode) *RangeNode {
return &RangeNode{BranchNode{tr: t, NodeType: NodeRange, Pos: pos, Line: line, Pipe: pipe, List: list, ElseList: elseList}}
}
func (r *RangeNode) Copy() Node {
return r.tr.newRange(r.Pos, r.Line, r.Pipe.CopyPipe(), r.List.CopyList(), r.ElseList.CopyList())
}
// WithNode represents a {{with}} action and its commands.
type WithNode struct {
BranchNode
}
func (t *Tree) newWith(pos Pos, line int, pipe *PipeNode, list, elseList *ListNode) *WithNode {
return &WithNode{BranchNode{tr: t, NodeType: NodeWith, Pos: pos, Line: line, Pipe: pipe, List: list, ElseList: elseList}}
}
func (w *WithNode) Copy() Node {
return w.tr.newWith(w.Pos, w.Line, w.Pipe.CopyPipe(), w.List.CopyList(), w.ElseList.CopyList())
}
// TemplateNode represents a {{template}} action.
type TemplateNode struct {
NodeType
Pos
tr *Tree
Line int // The line number in the input. Deprecated: Kept for compatibility.
Name string // The name of the template (unquoted).
Pipe *PipeNode // The command to evaluate as dot for the template.
}
func (t *Tree) newTemplate(pos Pos, line int, name string, pipe *PipeNode) *TemplateNode {
return &TemplateNode{tr: t, NodeType: NodeTemplate, Pos: pos, Line: line, Name: name, Pipe: pipe}
}
func (t *TemplateNode) String() string {
if t.Pipe == nil {
return fmt.Sprintf("{{template %q}}", t.Name)
}
return fmt.Sprintf("{{template %q %s}}", t.Name, t.Pipe)
}
func (t *TemplateNode) tree() *Tree {
return t.tr
}
func (t *TemplateNode) Copy() Node {
return t.tr.newTemplate(t.Pos, t.Line, t.Name, t.Pipe.CopyPipe())
}

View File

@ -0,0 +1,741 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Package parse builds parse trees for templates as defined by text/template
// and html/template. Clients should use those packages to construct templates
// rather than this one, which provides shared internal data structures not
// intended for general use.
package parse
import (
"bytes"
"fmt"
"runtime"
"strconv"
"strings"
)
// Tree is the representation of a single parsed template.
type Tree struct {
Name string // name of the template represented by the tree.
ParseName string // name of the top-level template during parsing, for error messages.
Root *ListNode // top-level root of the tree.
text string // text parsed to create the template (or its parent)
// Parsing only; cleared after parse.
funcs []map[string]interface{}
lex *lexer
token [3]item // three-token lookahead for parser.
peekCount int
vars []string // variables defined at the moment.
treeSet map[string]*Tree
}
// Copy returns a copy of the Tree. Any parsing state is discarded.
func (t *Tree) Copy() *Tree {
if t == nil {
return nil
}
return &Tree{
Name: t.Name,
ParseName: t.ParseName,
Root: t.Root.CopyList(),
text: t.text,
}
}
// Parse returns a map from template name to parse.Tree, created by parsing the
// templates described in the argument string. The top-level template will be
// given the specified name. If an error is encountered, parsing stops and an
// empty map is returned with the error.
func Parse(name, text, leftDelim, rightDelim string, funcs ...map[string]interface{}) (map[string]*Tree, error) {
treeSet := make(map[string]*Tree)
t := New(name)
t.text = text
_, err := t.Parse(text, leftDelim, rightDelim, treeSet, funcs...)
return treeSet, err
}
// next returns the next token.
func (t *Tree) next() item {
if t.peekCount > 0 {
t.peekCount--
} else {
t.token[0] = t.lex.nextItem()
}
return t.token[t.peekCount]
}
// backup backs the input stream up one token.
func (t *Tree) backup() {
t.peekCount++
}
// backup2 backs the input stream up two tokens.
// The zeroth token is already there.
func (t *Tree) backup2(t1 item) {
t.token[1] = t1
t.peekCount = 2
}
// backup3 backs the input stream up three tokens
// The zeroth token is already there.
func (t *Tree) backup3(t2, t1 item) { // Reverse order: we're pushing back.
t.token[1] = t1
t.token[2] = t2
t.peekCount = 3
}
// peek returns but does not consume the next token.
func (t *Tree) peek() item {
if t.peekCount > 0 {
return t.token[t.peekCount-1]
}
t.peekCount = 1
t.token[0] = t.lex.nextItem()
return t.token[0]
}
// nextNonSpace returns the next non-space token.
func (t *Tree) nextNonSpace() (token item) {
for {
token = t.next()
if token.typ != itemSpace {
break
}
}
return token
}
// peekNonSpace returns but does not consume the next non-space token.
func (t *Tree) peekNonSpace() (token item) {
for {
token = t.next()
if token.typ != itemSpace {
break
}
}
t.backup()
return token
}
// Parsing.
// New allocates a new parse tree with the given name.
func New(name string, funcs ...map[string]interface{}) *Tree {
return &Tree{
Name: name,
funcs: funcs,
}
}
// ErrorContext returns a textual representation of the location of the node in the input text.
// The receiver is only used when the node does not have a pointer to the tree inside,
// which can occur in old code.
func (t *Tree) ErrorContext(n Node) (location, context string) {
pos := int(n.Position())
tree := n.tree()
if tree == nil {
tree = t
}
text := tree.text[:pos]
byteNum := strings.LastIndex(text, "\n")
if byteNum == -1 {
byteNum = pos // On first line.
} else {
byteNum++ // After the newline.
byteNum = pos - byteNum
}
lineNum := 1 + strings.Count(text, "\n")
context = n.String()
if len(context) > 20 {
context = fmt.Sprintf("%.20s...", context)
}
return fmt.Sprintf("%s:%d:%d", tree.ParseName, lineNum, byteNum), context
}
// errorf formats the error and terminates processing.
func (t *Tree) errorf(format string, args ...interface{}) {
t.Root = nil
format = fmt.Sprintf("template: %s:%d: %s", t.ParseName, t.token[0].line, format)
panic(fmt.Errorf(format, args...))
}
// error terminates processing.
func (t *Tree) error(err error) {
t.errorf("%s", err)
}
// expect consumes the next token and guarantees it has the required type.
func (t *Tree) expect(expected itemType, context string) item {
token := t.nextNonSpace()
if token.typ != expected {
t.unexpected(token, context)
}
return token
}
// expectOneOf consumes the next token and guarantees it has one of the required types.
func (t *Tree) expectOneOf(expected1, expected2 itemType, context string) item {
token := t.nextNonSpace()
if token.typ != expected1 && token.typ != expected2 {
t.unexpected(token, context)
}
return token
}
// unexpected complains about the token and terminates processing.
func (t *Tree) unexpected(token item, context string) {
t.errorf("unexpected %s in %s", token, context)
}
// recover is the handler that turns panics into returns from the top level of Parse.
func (t *Tree) recover(errp *error) {
e := recover()
if e != nil {
if _, ok := e.(runtime.Error); ok {
panic(e)
}
if t != nil {
t.lex.drain()
t.stopParse()
}
*errp = e.(error)
}
}
// startParse initializes the parser, using the lexer.
func (t *Tree) startParse(funcs []map[string]interface{}, lex *lexer, treeSet map[string]*Tree) {
t.Root = nil
t.lex = lex
t.vars = []string{"$"}
t.funcs = funcs
t.treeSet = treeSet
}
// stopParse terminates parsing.
func (t *Tree) stopParse() {
t.lex = nil
t.vars = nil
t.funcs = nil
t.treeSet = nil
}
// Parse parses the template definition string to construct a representation of
// the template for execution. If either action delimiter string is empty, the
// default ("{{" or "}}") is used. Embedded template definitions are added to
// the treeSet map.
func (t *Tree) Parse(text, leftDelim, rightDelim string, treeSet map[string]*Tree, funcs ...map[string]interface{}) (tree *Tree, err error) {
defer t.recover(&err)
t.ParseName = t.Name
t.startParse(funcs, lex(t.Name, text, leftDelim, rightDelim), treeSet)
t.text = text
t.parse()
t.add()
t.stopParse()
return t, nil
}
// add adds tree to t.treeSet.
func (t *Tree) add() {
tree := t.treeSet[t.Name]
if tree == nil || IsEmptyTree(tree.Root) {
t.treeSet[t.Name] = t
return
}
if !IsEmptyTree(t.Root) {
t.errorf("template: multiple definition of template %q", t.Name)
}
}
// IsEmptyTree reports whether this tree (node) is empty of everything but space.
func IsEmptyTree(n Node) bool {
switch n := n.(type) {
case nil:
return true
case *ActionNode:
case *IfNode:
case *ListNode:
for _, node := range n.Nodes {
if !IsEmptyTree(node) {
return false
}
}
return true
case *RangeNode:
case *TemplateNode:
case *TextNode:
return len(bytes.TrimSpace(n.Text)) == 0
case *WithNode:
default:
panic("unknown node: " + n.String())
}
return false
}
// parse is the top-level parser for a template, essentially the same
// as itemList except it also parses {{define}} actions.
// It runs to EOF.
func (t *Tree) parse() {
t.Root = t.newList(t.peek().pos)
for t.peek().typ != itemEOF {
if t.peek().typ == itemLeftDelim {
delim := t.next()
if t.nextNonSpace().typ == itemDefine {
newT := New("definition") // name will be updated once we know it.
newT.text = t.text
newT.ParseName = t.ParseName
newT.startParse(t.funcs, t.lex, t.treeSet)
newT.parseDefinition()
continue
}
t.backup2(delim)
}
switch n := t.textOrAction(); n.Type() {
case nodeEnd, nodeElse:
t.errorf("unexpected %s", n)
default:
t.Root.append(n)
}
}
}
// parseDefinition parses a {{define}} ... {{end}} template definition and
// installs the definition in t.treeSet. The "define" keyword has already
// been scanned.
func (t *Tree) parseDefinition() {
const context = "define clause"
name := t.expectOneOf(itemString, itemRawString, context)
var err error
t.Name, err = strconv.Unquote(name.val)
if err != nil {
t.error(err)
}
t.expect(itemRightDelim, context)
var end Node
t.Root, end = t.itemList()
if end.Type() != nodeEnd {
t.errorf("unexpected %s in %s", end, context)
}
t.add()
t.stopParse()
}
// itemList:
// textOrAction*
// Terminates at {{end}} or {{else}}, returned separately.
func (t *Tree) itemList() (list *ListNode, next Node) {
list = t.newList(t.peekNonSpace().pos)
for t.peekNonSpace().typ != itemEOF {
n := t.textOrAction()
switch n.Type() {
case nodeEnd, nodeElse:
return list, n
}
list.append(n)
}
t.errorf("unexpected EOF")
return
}
// textOrAction:
// text | action
func (t *Tree) textOrAction() Node {
switch token := t.nextNonSpace(); token.typ {
case itemText:
return t.newText(token.pos, token.val)
case itemLeftDelim:
return t.action()
default:
t.unexpected(token, "input")
}
return nil
}
// Action:
// control
// command ("|" command)*
// Left delim is past. Now get actions.
// First word could be a keyword such as range.
func (t *Tree) action() (n Node) {
switch token := t.nextNonSpace(); token.typ {
case itemBlock:
return t.blockControl()
case itemElse:
return t.elseControl()
case itemEnd:
return t.endControl()
case itemIf:
return t.ifControl()
case itemRange:
return t.rangeControl()
case itemTemplate:
return t.templateControl()
case itemWith:
return t.withControl()
}
t.backup()
token := t.peek()
// Do not pop variables; they persist until "end".
return t.newAction(token.pos, token.line, t.pipeline("command"))
}
// Pipeline:
// declarations? command ('|' command)*
func (t *Tree) pipeline(context string) (pipe *PipeNode) {
decl := false
var vars []*VariableNode
token := t.peekNonSpace()
pos := token.pos
// Are there declarations or assignments?
for {
if v := t.peekNonSpace(); v.typ == itemVariable {
t.next()
// Since space is a token, we need 3-token look-ahead here in the worst case:
// in "$x foo" we need to read "foo" (as opposed to ":=") to know that $x is an
// argument variable rather than a declaration. So remember the token
// adjacent to the variable so we can push it back if necessary.
tokenAfterVariable := t.peek()
next := t.peekNonSpace()
switch {
case next.typ == itemAssign, next.typ == itemDeclare,
next.typ == itemChar && next.val == ",":
t.nextNonSpace()
variable := t.newVariable(v.pos, v.val)
vars = append(vars, variable)
t.vars = append(t.vars, v.val)
if next.typ == itemDeclare {
decl = true
}
if next.typ == itemChar && next.val == "," {
if context == "range" && len(vars) < 2 {
continue
}
t.errorf("too many declarations in %s", context)
}
case tokenAfterVariable.typ == itemSpace:
t.backup3(v, tokenAfterVariable)
default:
t.backup2(v)
}
}
break
}
pipe = t.newPipeline(pos, token.line, vars)
pipe.IsAssign = !decl
for {
switch token := t.nextNonSpace(); token.typ {
case itemRightDelim, itemRightParen:
// At this point, the pipeline is complete
t.checkPipeline(pipe, context)
if token.typ == itemRightParen {
t.backup()
}
return
case itemBool, itemCharConstant, itemComplex, itemDot, itemField, itemIdentifier,
itemNumber, itemNil, itemRawString, itemString, itemVariable, itemLeftParen:
t.backup()
pipe.append(t.command())
default:
t.unexpected(token, context)
}
}
}
func (t *Tree) checkPipeline(pipe *PipeNode, context string) {
// Reject empty pipelines
if len(pipe.Cmds) == 0 {
t.errorf("missing value for %s", context)
}
// Only the first command of a pipeline can start with a non executable operand
for i, c := range pipe.Cmds[1:] {
switch c.Args[0].Type() {
case NodeBool, NodeDot, NodeNil, NodeNumber, NodeString:
// With A|B|C, pipeline stage 2 is B
t.errorf("non executable command in pipeline stage %d", i+2)
}
}
}
func (t *Tree) parseControl(allowElseIf bool, context string) (pos Pos, line int, pipe *PipeNode, list, elseList *ListNode) {
defer t.popVars(len(t.vars))
pipe = t.pipeline(context)
var next Node
list, next = t.itemList()
switch next.Type() {
case nodeEnd: //done
case nodeElse:
if allowElseIf {
// Special case for "else if". If the "else" is followed immediately by an "if",
// the elseControl will have left the "if" token pending. Treat
// {{if a}}_{{else if b}}_{{end}}
// as
// {{if a}}_{{else}}{{if b}}_{{end}}{{end}}.
// To do this, parse the if as usual and stop at it {{end}}; the subsequent{{end}}
// is assumed. This technique works even for long if-else-if chains.
// TODO: Should we allow else-if in with and range?
if t.peek().typ == itemIf {
t.next() // Consume the "if" token.
elseList = t.newList(next.Position())
elseList.append(t.ifControl())
// Do not consume the next item - only one {{end}} required.
break
}
}
elseList, next = t.itemList()
if next.Type() != nodeEnd {
t.errorf("expected end; found %s", next)
}
}
return pipe.Position(), pipe.Line, pipe, list, elseList
}
// If:
// {{if pipeline}} itemList {{end}}
// {{if pipeline}} itemList {{else}} itemList {{end}}
// If keyword is past.
func (t *Tree) ifControl() Node {
return t.newIf(t.parseControl(true, "if"))
}
// Range:
// {{range pipeline}} itemList {{end}}
// {{range pipeline}} itemList {{else}} itemList {{end}}
// Range keyword is past.
func (t *Tree) rangeControl() Node {
return t.newRange(t.parseControl(false, "range"))
}
// With:
// {{with pipeline}} itemList {{end}}
// {{with pipeline}} itemList {{else}} itemList {{end}}
// If keyword is past.
func (t *Tree) withControl() Node {
return t.newWith(t.parseControl(false, "with"))
}
// End:
// {{end}}
// End keyword is past.
func (t *Tree) endControl() Node {
return t.newEnd(t.expect(itemRightDelim, "end").pos)
}
// Else:
// {{else}}
// Else keyword is past.
func (t *Tree) elseControl() Node {
// Special case for "else if".
peek := t.peekNonSpace()
if peek.typ == itemIf {
// We see "{{else if ... " but in effect rewrite it to {{else}}{{if ... ".
return t.newElse(peek.pos, peek.line)
}
token := t.expect(itemRightDelim, "else")
return t.newElse(token.pos, token.line)
}
// Block:
// {{block stringValue pipeline}}
// Block keyword is past.
// The name must be something that can evaluate to a string.
// The pipeline is mandatory.
func (t *Tree) blockControl() Node {
const context = "block clause"
token := t.nextNonSpace()
name := t.parseTemplateName(token, context)
pipe := t.pipeline(context)
block := New(name) // name will be updated once we know it.
block.text = t.text
block.ParseName = t.ParseName
block.startParse(t.funcs, t.lex, t.treeSet)
var end Node
block.Root, end = block.itemList()
if end.Type() != nodeEnd {
t.errorf("unexpected %s in %s", end, context)
}
block.add()
block.stopParse()
return t.newTemplate(token.pos, token.line, name, pipe)
}
// Template:
// {{template stringValue pipeline}}
// Template keyword is past. The name must be something that can evaluate
// to a string.
func (t *Tree) templateControl() Node {
const context = "template clause"
token := t.nextNonSpace()
name := t.parseTemplateName(token, context)
var pipe *PipeNode
if t.nextNonSpace().typ != itemRightDelim {
t.backup()
// Do not pop variables; they persist until "end".
pipe = t.pipeline(context)
}
return t.newTemplate(token.pos, token.line, name, pipe)
}
func (t *Tree) parseTemplateName(token item, context string) (name string) {
switch token.typ {
case itemString, itemRawString:
s, err := strconv.Unquote(token.val)
if err != nil {
t.error(err)
}
name = s
default:
t.unexpected(token, context)
}
return
}
// command:
// operand (space operand)*
// space-separated arguments up to a pipeline character or right delimiter.
// we consume the pipe character but leave the right delim to terminate the action.
func (t *Tree) command() *CommandNode {
cmd := t.newCommand(t.peekNonSpace().pos)
for {
t.peekNonSpace() // skip leading spaces.
operand := t.operand()
if operand != nil {
cmd.append(operand)
}
switch token := t.next(); token.typ {
case itemSpace:
continue
case itemError:
t.errorf("%s", token.val)
case itemRightDelim, itemRightParen:
t.backup()
case itemPipe:
default:
t.errorf("unexpected %s in operand", token)
}
break
}
if len(cmd.Args) == 0 {
t.errorf("empty command")
}
return cmd
}
// operand:
// term .Field*
// An operand is a space-separated component of a command,
// a term possibly followed by field accesses.
// A nil return means the next item is not an operand.
func (t *Tree) operand() Node {
node := t.term()
if node == nil {
return nil
}
if t.peek().typ == itemField {
chain := t.newChain(t.peek().pos, node)
for t.peek().typ == itemField {
chain.Add(t.next().val)
}
// Compatibility with original API: If the term is of type NodeField
// or NodeVariable, just put more fields on the original.
// Otherwise, keep the Chain node.
// Obvious parsing errors involving literal values are detected here.
// More complex error cases will have to be handled at execution time.
switch node.Type() {
case NodeField:
node = t.newField(chain.Position(), chain.String())
case NodeVariable:
node = t.newVariable(chain.Position(), chain.String())
case NodeBool, NodeString, NodeNumber, NodeNil, NodeDot:
t.errorf("unexpected . after term %q", node.String())
default:
node = chain
}
}
return node
}
// term:
// literal (number, string, nil, boolean)
// function (identifier)
// .
// .Field
// $
// '(' pipeline ')'
// A term is a simple "expression".
// A nil return means the next item is not a term.
func (t *Tree) term() Node {
switch token := t.nextNonSpace(); token.typ {
case itemError:
t.errorf("%s", token.val)
case itemIdentifier:
if !t.hasFunction(token.val) {
t.errorf("function %q not defined", token.val)
}
return NewIdentifier(token.val).SetTree(t).SetPos(token.pos)
case itemDot:
return t.newDot(token.pos)
case itemNil:
return t.newNil(token.pos)
case itemVariable:
return t.useVar(token.pos, token.val)
case itemField:
return t.newField(token.pos, token.val)
case itemBool:
return t.newBool(token.pos, token.val == "true")
case itemCharConstant, itemComplex, itemNumber:
number, err := t.newNumber(token.pos, token.val, token.typ)
if err != nil {
t.error(err)
}
return number
case itemLeftParen:
pipe := t.pipeline("parenthesized pipeline")
if token := t.next(); token.typ != itemRightParen {
t.errorf("unclosed right paren: unexpected %s", token)
}
return pipe
case itemString, itemRawString:
s, err := strconv.Unquote(token.val)
if err != nil {
t.error(err)
}
return t.newString(token.pos, token.val, s)
}
t.backup()
return nil
}
// hasFunction reports if a function name exists in the Tree's maps.
func (t *Tree) hasFunction(name string) bool {
for _, funcMap := range t.funcs {
if funcMap == nil {
continue
}
if funcMap[name] != nil {
return true
}
}
return false
}
// popVars trims the variable list to the specified length
func (t *Tree) popVars(n int) {
t.vars = t.vars[:n]
}
// useVar returns a node for a variable reference. It errors if the
// variable is not defined.
func (t *Tree) useVar(pos Pos, name string) Node {
v := t.newVariable(pos, name)
for _, varName := range t.vars {
if varName == v.Ident[0] {
return v
}
}
t.errorf("undefined variable %q", v.Ident[0])
return nil
}

View File

@ -0,0 +1,224 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package template
import (
"reflect"
"sync"
"gitee.com/johng/gf/g/os/gview/internal/text/template/parse"
)
// common holds the information shared by related templates.
type common struct {
tmpl map[string]*Template // Map from name to defined templates.
option option
// We use two maps, one for parsing and one for execution.
// This separation makes the API cleaner since it doesn't
// expose reflection to the client.
muFuncs sync.RWMutex // protects parseFuncs and execFuncs
parseFuncs FuncMap
execFuncs map[string]reflect.Value
}
// Template is the representation of a parsed template. The *parse.Tree
// field is exported only for use by html/template and should be treated
// as unexported by all other clients.
type Template struct {
name string
*parse.Tree
*common
leftDelim string
rightDelim string
}
// New allocates a new, undefined template with the given name.
func New(name string) *Template {
t := &Template{
name: name,
}
t.init()
return t
}
// Name returns the name of the template.
func (t *Template) Name() string {
return t.name
}
// New allocates a new, undefined template associated with the given one and with the same
// delimiters. The association, which is transitive, allows one template to
// invoke another with a {{template}} action.
func (t *Template) New(name string) *Template {
t.init()
nt := &Template{
name: name,
common: t.common,
leftDelim: t.leftDelim,
rightDelim: t.rightDelim,
}
return nt
}
// init guarantees that t has a valid common structure.
func (t *Template) init() {
if t.common == nil {
c := new(common)
c.tmpl = make(map[string]*Template)
c.parseFuncs = make(FuncMap)
c.execFuncs = make(map[string]reflect.Value)
t.common = c
}
}
// Clone returns a duplicate of the template, including all associated
// templates. The actual representation is not copied, but the name space of
// associated templates is, so further calls to Parse in the copy will add
// templates to the copy but not to the original. Clone can be used to prepare
// common templates and use them with variant definitions for other templates
// by adding the variants after the clone is made.
func (t *Template) Clone() (*Template, error) {
nt := t.copy(nil)
nt.init()
if t.common == nil {
return nt, nil
}
for k, v := range t.tmpl {
if k == t.name {
nt.tmpl[t.name] = nt
continue
}
// The associated templates share nt's common structure.
tmpl := v.copy(nt.common)
nt.tmpl[k] = tmpl
}
t.muFuncs.RLock()
defer t.muFuncs.RUnlock()
for k, v := range t.parseFuncs {
nt.parseFuncs[k] = v
}
for k, v := range t.execFuncs {
nt.execFuncs[k] = v
}
return nt, nil
}
// copy returns a shallow copy of t, with common set to the argument.
func (t *Template) copy(c *common) *Template {
nt := New(t.name)
nt.Tree = t.Tree
nt.common = c
nt.leftDelim = t.leftDelim
nt.rightDelim = t.rightDelim
return nt
}
// AddParseTree adds parse tree for template with given name and associates it with t.
// If the template does not already exist, it will create a new one.
// If the template does exist, it will be replaced.
func (t *Template) AddParseTree(name string, tree *parse.Tree) (*Template, error) {
t.init()
// If the name is the name of this template, overwrite this template.
nt := t
if name != t.name {
nt = t.New(name)
}
// Even if nt == t, we need to install it in the common.tmpl map.
if t.associate(nt, tree) || nt.Tree == nil {
nt.Tree = tree
}
return nt, nil
}
// Templates returns a slice of defined templates associated with t.
func (t *Template) Templates() []*Template {
if t.common == nil {
return nil
}
// Return a slice so we don't expose the map.
m := make([]*Template, 0, len(t.tmpl))
for _, v := range t.tmpl {
m = append(m, v)
}
return m
}
// Delims sets the action delimiters to the specified strings, to be used in
// subsequent calls to Parse, ParseFiles, or ParseGlob. Nested template
// definitions will inherit the settings. An empty delimiter stands for the
// corresponding default: {{ or }}.
// The return value is the template, so calls can be chained.
func (t *Template) Delims(left, right string) *Template {
t.init()
t.leftDelim = left
t.rightDelim = right
return t
}
// Funcs adds the elements of the argument map to the template's function map.
// It must be called before the template is parsed.
// It panics if a value in the map is not a function with appropriate return
// type or if the name cannot be used syntactically as a function in a template.
// It is legal to overwrite elements of the map. The return value is the template,
// so calls can be chained.
func (t *Template) Funcs(funcMap FuncMap) *Template {
t.init()
t.muFuncs.Lock()
defer t.muFuncs.Unlock()
addValueFuncs(t.execFuncs, funcMap)
addFuncs(t.parseFuncs, funcMap)
return t
}
// Lookup returns the template with the given name that is associated with t.
// It returns nil if there is no such template or the template has no definition.
func (t *Template) Lookup(name string) *Template {
if t.common == nil {
return nil
}
return t.tmpl[name]
}
// Parse parses text as a template body for t.
// Named template definitions ({{define ...}} or {{block ...}} statements) in text
// define additional templates associated with t and are removed from the
// definition of t itself.
//
// Templates can be redefined in successive calls to Parse.
// A template definition with a body containing only white space and comments
// is considered empty and will not replace an existing template's body.
// This allows using Parse to add new named template definitions without
// overwriting the main template body.
func (t *Template) Parse(text string) (*Template, error) {
t.init()
t.muFuncs.RLock()
trees, err := parse.Parse(t.name, text, t.leftDelim, t.rightDelim, t.parseFuncs, builtins)
t.muFuncs.RUnlock()
if err != nil {
return nil, err
}
// Add the newly parsed trees, including the one for t, into our common structure.
for name, tree := range trees {
if _, err := t.AddParseTree(name, tree); err != nil {
return nil, err
}
}
return t, nil
}
// associate installs the new template into the group of templates associated
// with t. The two are already known to share the common structure.
// The boolean return value reports whether to store this tree as t.Tree.
func (t *Template) associate(new *Template, tree *parse.Tree) bool {
if new.common != t.common {
panic("internal error: associate not common")
}
if old := t.tmpl[new.name]; old != nil && parse.IsEmptyTree(tree.Root) && old.Tree != nil {
// If a template by that name exists,
// don't replace it with an empty template.
return false
}
t.tmpl[new.name] = new
return true
}

View File

@ -0,0 +1,3 @@
// from golang-1.11.2
// 1. remove "<no value>" when template variable does not exist;
package text

113
g/util/gconv/gconv_map.go Normal file
View File

@ -0,0 +1,113 @@
// Copyright 2018 gf Author(https://gitee.com/johng/gf). All Rights Reserved.
//
// This Source Code Form is subject to the terms of the MIT License.
// If a copy of the MIT was not distributed with this file,
// You can obtain one at https://gitee.com/johng/gf.
package gconv
import (
"reflect"
)
// 任意类型转换为 map[string]interface{} 类型,
// 如果给定的输入参数i不是map类型那么转换会失败返回nil.
// 当i为struct对象时第二个参数noTagCheck表示不检测json标签否则将会使用json tag作为map的键名。
func Map(i interface{}, noTagCheck...bool) map[string]interface{} {
if i == nil {
return nil
}
if r, ok := i.(map[string]interface{}); ok {
return r
} else {
// 仅对常见的几种map组合进行断言最后才会使用反射
m := make(map[string]interface{})
switch i.(type) {
case map[interface{}]interface{}:
for k, v := range i.(map[interface{}]interface{}) {
m[String(k)] = v
}
case map[interface{}]string:
for k, v := range i.(map[interface{}]string) {
m[String(k)] = v
}
case map[interface{}]int:
for k, v := range i.(map[interface{}]int) {
m[String(k)] = v
}
case map[interface{}]uint:
for k, v := range i.(map[interface{}]uint) {
m[String(k)] = v
}
case map[interface{}]float32:
for k, v := range i.(map[interface{}]float32) {
m[String(k)] = v
}
case map[interface{}]float64:
for k, v := range i.(map[interface{}]float64) {
m[String(k)] = v
}
case map[string]bool:
for k, v := range i.(map[string]bool) {
m[k] = v
}
case map[string]int:
for k, v := range i.(map[string]int) {
m[k] = v
}
case map[string]uint:
for k, v := range i.(map[string]uint) {
m[k] = v
}
case map[string]float32:
for k, v := range i.(map[string]float32) {
m[k] = v
}
case map[string]float64:
for k, v := range i.(map[string]float64) {
m[k] = v
}
case map[int]interface{}:
for k, v := range i.(map[int]interface{}) {
m[String(k)] = v
}
case map[int]string:
for k, v := range i.(map[int]string) {
m[String(k)] = v
}
case map[uint]string:
for k, v := range i.(map[uint]string) {
m[String(k)] = v
}
// 不是常见类型,则使用反射
default:
rv := reflect.ValueOf(i)
kind := rv.Kind()
// 如果是指针,那么需要转换到指针对应的数据项,以便识别真实的类型
if kind == reflect.Ptr {
rv = rv.Elem()
kind = rv.Kind()
}
if kind == reflect.Map {
ks := rv.MapKeys()
for _, k := range ks {
m[String(k.Interface())] = rv.MapIndex(k).Interface()
}
} else if kind == reflect.Struct {
rt := rv.Type()
name := ""
for i := 0; i < rv.NumField(); i++ {
if name = rt.Field(i).Tag.Get("json"); name == "" {
name = rt.Field(i).Name
}
m[name] = rv.Field(i).Interface()
}
} else {
return nil
}
}
return m
}
}

View File

@ -6,7 +6,10 @@
package gconv
import "fmt"
import (
"fmt"
"reflect"
)
// 任意类型转换为[]int类型
func Ints(i interface{}) []int {
@ -296,6 +299,26 @@ func Interfaces(i interface{}) []interface{} {
for _, v := range i.([]float64) {
array = append(array, v)
}
// 不是常见类型,则使用反射
default:
rv := reflect.ValueOf(i)
kind := rv.Kind()
// 如果是指针,那么需要转换到指针对应的数据项,以便识别真实的类型
if kind == reflect.Ptr {
rv = rv.Elem()
kind = rv.Kind()
}
switch kind {
case reflect.Slice: fallthrough
case reflect.Array:
for i := 0; i < rv.Len(); i++ {
array = append(array, rv.Index(i).Interface())
}
case reflect.Struct:
for i := 0; i < rv.NumField(); i++ {
array = append(array, rv.Field(i).Interface())
}
}
}
if len(array) > 0 {
return array

View File

@ -205,10 +205,12 @@ func bindVarToStructIfDefaultConvertionFailed(structFieldValue reflect.Value, va
switch structFieldValue.Kind() {
case reflect.Struct:
Struct(value, structFieldValue)
case reflect.Slice:
case reflect.Slice: fallthrough
case reflect.Array:
a := reflect.Value{}
v := reflect.ValueOf(value)
if v.Kind() == reflect.Slice {
if v.Kind() == reflect.Slice || v.Kind() == reflect.Array {
a = reflect.MakeSlice(structFieldValue.Type(), v.Len(), v.Len())
for i := 0; i < v.Len(); i++ {
n := reflect.New(structFieldValue.Type().Elem()).Elem()

View File

@ -9,6 +9,7 @@ package grand
import (
"crypto/rand"
"encoding/binary"
"time"
)
const (
@ -21,14 +22,23 @@ var (
// 使用缓冲区实现快速的随机数生成
func init() {
step := 0
buffer := make([]byte, 1024)
go func() {
for {
if n, err := rand.Read(buffer); err != nil {
panic(err)
} else {
for i := 0; i < n - 8; i += 8 {
// 使用缓冲区数据进行一次完整的随机数生成
for i := 0; i < n - 8; {
bufferChan <- binary.LittleEndian.Uint64(buffer[i : i + 8])
i ++
}
// 充分利用缓冲区数据,字节倒序生成,随机索引递增
step = int(time.Now().UnixNano())%n
for i := 0; i < n - 8; {
bufferChan <- binary.BigEndian.Uint64(buffer[i : i + 8])
i += step
}
}
}

View File

@ -9,12 +9,22 @@
package grand_test
import (
"testing"
"gitee.com/johng/gf/g/util/grand"
"testing"
)
var buffer = make([]byte, 8)
func Benchmark_Rand(b *testing.B) {
for i := 0; i < b.N; i++ {
grand.Rand(0, 999999999)
}
}
//func Benchmark_Buffer(b *testing.B) {
// for i := 0; i < b.N; i++ {
// if _, err := rand.Read(buffer); err == nil {
// binary.LittleEndian.Uint64(buffer)
// }
// }
//}

34
g/util/gstr/gstr_test.go Normal file
View File

@ -0,0 +1,34 @@
// Copyright 2018 gf Author(https://gitee.com/johng/gf). All Rights Reserved.
//
// This Source Code Form is subject to the terms of the MIT License.
// If a copy of the MIT was not distributed with this file,
// You can obtain one at https://gitee.com/johng/gf.
// go test *.go -bench=".*"
package gstr_test
import (
"testing"
)
var (
str = "This is the test string for gstr."
bytes = []byte(str)
)
func Benchmark_StringToBytes(b *testing.B) {
for i := 0; i < b.N; i++ {
if []byte(str) != nil {
}
}
}
func Benchmark_BytesToString(b *testing.B) {
for i := 0; i < b.N; i++ {
if string(bytes) != "" {
}
}
}

View File

@ -5,7 +5,7 @@
// You can obtain one at https://gitee.com/johng/gf.
// 单元测试
// go test *.go -bench=".*"
// go test *.go
package gvalid

View File

@ -0,0 +1,17 @@
package main
import "fmt"
func main() {
array := []uint{1, 1, 2, 3, 3, 4, 4, 5, 5, 6, 6}
for i := 0; i < len(array) - 1; i++ {
for j := i + 1; j < len(array); j++ {
if array[i] == array[j] {
array = append(array[ : j], array[j + 1 : ]...)
}
}
}
fmt.Println(array)
}

View File

@ -1,4 +1,3 @@
// 验证 map 的delete方法是否并发安全
package main
import (
@ -7,6 +6,7 @@ import (
"fmt"
)
// 验证 map 的delete方法是否并发安全
func main() {
// 创建一个初始化的map
m := make(map[int]int)

View File

@ -0,0 +1,28 @@
package main
import (
"fmt"
"time"
"gitee.com/johng/gf/g/os/gtime"
"gitee.com/johng/gf/g/container/gqueue"
)
func main() {
q := gqueue.New()
// 数据生产者每隔1秒往队列写数据
gtime.SetInterval(time.Second, func() bool {
for i := 0; i < 10; i++ {
q.Push(i)
}
return true
})
// 消费者,不停读取队列数据并输出到终端
for {
if v := q.Pop(); v != nil {
fmt.Println(" Pop:", v)
} else {
break
}
}
}

View File

@ -29,4 +29,8 @@ func main() {
s := new(Score)
v.Struct(s)
fmt.Println(s)
// 只读接口
r := v.ReadOnly()
fmt.Println(r.String())
}

View File

@ -0,0 +1,582 @@
package main
import (
"fmt"
"time"
//_ "github.com/denisenkom/go-mssqldb"
"gitee.com/johng/gf/g/database/gdb"
"gitee.com/johng/gf/g"
)
// 本文件用于gf框架的mssql数据库操作示例不作为单元测试使用
var db *gdb.Db
// 初始化配置及创建数据库
func init () {
gdb.AddDefaultConfigNode(gdb.ConfigNode {
Host : "127.0.0.1",
Port : "1433",
User : "test",
Pass : "test",
Name : "test",
Type : "mssql",
Role : "master",
Charset : "utf8",
})
db, _= gdb.New()
//gins.Config().SetPath("/home/john/Workspace/Go/GOPATH/src/gitee.com/johng/gf/geg/frame")
//db = g.Database()
//gdb.SetConfig(gdb.ConfigNode {
// Host : "127.0.0.1",
// Port : 3306,
// User : "root",
// Pass : "123456",
// Name : "test",
// Type : "mysql",
//})
//db, _ = gdb.Instance()
//gdb.SetConfig(gdb.Config {
// "default" : gdb.ConfigGroup {
// gdb.ConfigNode {
// Host : "127.0.0.1",
// Port : "3306",
// User : "root",
// Pass : "123456",
// Name : "test",
// Type : "mysql",
// Role : "master",
// Priority : 100,
// },
// gdb.ConfigNode {
// Host : "127.0.0.2",
// Port : "3306",
// User : "root",
// Pass : "123456",
// Name : "test",
// Type : "mysql",
// Role : "master",
// Priority : 100,
// },
// gdb.ConfigNode {
// Host : "127.0.0.3",
// Port : "3306",
// User : "root",
// Pass : "123456",
// Name : "test",
// Type : "mysql",
// Role : "master",
// Priority : 100,
// },
// gdb.ConfigNode {
// Host : "127.0.0.4",
// Port : "3306",
// User : "root",
// Pass : "123456",
// Name : "test",
// Type : "mysql",
// Role : "master",
// Priority : 100,
// },
// },
//})
//db, _ = gdb.Instance()
}
// 创建测试数据库
func create() error {
fmt.Println("drop table aa_user:")
_, err := db.Exec("drop table aa_user")
if err != nil {
fmt.Println("drop table aa_user error.",err)
}
s := `
CREATE TABLE aa_user (
id int not null,
name VARCHAR(60),
age int,
addr varchar(60),
PRIMARY KEY (id)
)
`
fmt.Println("create table aa_user:")
_, err = db.Exec(s)
if err != nil {
fmt.Println("create table error.",err)
return err
}
/*_, err = db.Exec("drop sequence id_seq")
if err != nil {
fmt.Println("drop sequence id_seq", err)
}
fmt.Println("create sequence id_seq")
_, err = db.Exec("create sequence id_seq increment by 1 start with 1 maxvalue 9999999999 cycle cache 10")
if err != nil {
fmt.Println("create sequence id_seq error.", err)
return err
}
s = `
CREATE TRIGGER id_trigger before insert on aa_user for each row
begin
select id_seq.nextval into :new.id from dual;
end;
`
_, err = db.Exec(s)
if err != nil {
fmt.Println("create trigger error.", err)
return err
}*/
_, err = db.Exec("drop table user_detail")
if err != nil {
fmt.Println("drop table user_detail", err)
}
s = `
CREATE TABLE user_detail (
id int not null,
site VARCHAR(255),
PRIMARY KEY (id)
)
`
fmt.Println("create table user_detail:")
_, err = db.Exec(s)
if err != nil {
fmt.Println("create table user_detail error.",err)
return err
}
fmt.Println("create table success.")
return nil
}
// 数据写入
func insert(id int) {
fmt.Println("insert:")
r, err := db.Insert("aa_user", gdb.Map {
"id": id,
"name": "john",
"age": id,
})
fmt.Println(r.LastInsertId())
fmt.Println(r.RowsAffected())
if err == nil {
r, err = db.Insert("user_detail", gdb.Map {
"id" : id,
"site" : "http://johng.cn",
})
if err == nil {
fmt.Printf("id: %d\n", id)
} else {
fmt.Println(err)
}
} else {
fmt.Println(err)
}
fmt.Println()
}
// 基本sql查询
func query() {
fmt.Println("query:")
list, err := db.GetAll("select * from aa_user where id='1'")
if err == nil {
fmt.Println(list)
} else {
fmt.Println(err)
}
list, err = db.Table("aa_user").OrderBy("id").Limit(0,2).Select()
if err == nil {
fmt.Println(list)
} else {
fmt.Println(err)
}
fmt.Println()
}
// replace into
func replace() {
fmt.Println("replace:")
r, err := db.Save("aa_user", gdb.Map {
"id" : 1,
"name" : "john",
})
if err == nil {
fmt.Println(r.LastInsertId())
fmt.Println(r.RowsAffected())
} else {
fmt.Println(err)
}
fmt.Println()
}
// 数据保存
func save() {
fmt.Println("save:")
r, err := db.Save("aa_user", gdb.Map {
"id" : 1,
"name" : "john",
})
if err == nil {
fmt.Println(r.LastInsertId())
fmt.Println(r.RowsAffected())
} else {
fmt.Println(err)
}
fmt.Println()
}
// 批量写入
func batchInsert() {
fmt.Println("batchInsert:")
_, err := db.BatchInsert("aa_user", gdb.List {
{"id":11,"name": "batchInsert_john_1", "age": 11},
{"id":12,"name": "batchInsert_john_2", "age": 12},
{"id":13,"name": "batchInsert_john_3", "age": 13},
{"id":14,"name": "batchInsert_john_4", "age": 14},
}, 10)
if err != nil {
fmt.Println(err)
}
fmt.Println()
}
// 数据更新
func update1() {
fmt.Println("update1:")
r, err := db.Update("aa_user", gdb.Map {"name": "john1","age":1}, "id=?", 1)
if err == nil {
fmt.Println(r.LastInsertId())
fmt.Println(r.RowsAffected())
} else {
fmt.Println(err)
}
fmt.Println()
}
// 数据更新
func update2() {
fmt.Println("update2:")
r, err := db.Update("aa_user", gdb.Map{"name" : "john6","age":6}, "id=?", 2)
if err == nil {
fmt.Println(r.LastInsertId())
fmt.Println(r.RowsAffected())
} else {
fmt.Println(err)
}
fmt.Println()
}
// 数据更新
func update3() {
fmt.Println("update3:")
r, err := db.Update("aa_user", "name=?", "id=?", "john2", 3)
if err == nil {
fmt.Println(r.LastInsertId())
fmt.Println(r.RowsAffected())
} else {
fmt.Println(err)
}
fmt.Println()
}
// 链式查询操作1
func linkopSelect1() {
fmt.Println("linkopSelect1:")
r, err := db.Table("aa_user u").LeftJoin("user_detail ud", "u.id=ud.id").Fields("u.*, ud.site").Where("u.id > ?", 1).Limit(3, 5).Select()
if err == nil {
fmt.Println(r)
} else {
fmt.Println(err)
}
fmt.Println()
}
// 链式查询操作2
func linkopSelect2() {
fmt.Println("linkopSelect2:")
r, err := db.Table("aa_user u").LeftJoin("user_detail ud", "u.id=ud.id").Fields("u.*,ud.site").Where("u.id=?", 1).One()
if err == nil {
fmt.Println(r)
} else {
fmt.Println(err)
}
fmt.Println()
}
// 链式查询操作3
func linkopSelect3() {
fmt.Println("linkopSelect3:")
r, err := db.Table("aa_user u").LeftJoin("user_detail ud", "u.id=ud.id").Fields("ud.site").Where("u.id=?", 1).Value()
if err == nil {
fmt.Println(r.String())
} else {
fmt.Println(err)
}
fmt.Println()
}
// 链式查询数量1
func linkopCount1() {
fmt.Println("linkopCount1:")
r, err := db.Table("aa_user u").LeftJoin("user_detail ud", "u.id=ud.id").Where("name like ?", "john").Count()
if err == nil {
fmt.Println(r)
} else {
fmt.Println(err)
}
fmt.Println()
}
// 错误操作
func linkopUpdate1() {
fmt.Println("linkopUpdate1:")
r, err := db.Table("henghe_setting").Update()
if err == nil {
fmt.Println(r.RowsAffected())
} else {
fmt.Println("error",err)
}
fmt.Println()
}
// 通过Map指针方式传参方式
func linkopUpdate2() {
fmt.Println("linkopUpdate2:")
r, err := db.Table("aa_user").Data(gdb.Map{"name" : "john2"}).Where("name=?", "john").Update()
if err == nil {
fmt.Println(r.RowsAffected())
} else {
fmt.Println(err)
}
fmt.Println()
}
// 通过字符串方式传参
func linkopUpdate3() {
fmt.Println("linkopUpdate3:")
r, err := db.Table("aa_user").Data("name='john3'").Where("name=?", "john2").Update()
if err == nil {
fmt.Println(r.RowsAffected())
} else {
fmt.Println(err)
}
fmt.Println()
}
// Where条件使用Map
func linkopUpdate4() {
fmt.Println("linkopUpdate4:")
r, err := db.Table("aa_user").Data(gdb.Map{"name" : "john11111"}).Where(g.Map{"id" : 1}).Update()
if err == nil {
fmt.Println(r.RowsAffected())
} else {
fmt.Println(err)
}
fmt.Println()
}
// 链式批量写入
func linkopBatchInsert1() {
fmt.Println("linkopBatchInsert1:")
r, err := db.Table("aa_user").Data(gdb.List{
{"id":21,"name": "linkopBatchInsert1_john_1"},
{"id":22,"name": "linkopBatchInsert1_john_2"},
{"id":23,"name": "linkopBatchInsert1_john_3"},
{"id":24,"name": "linkopBatchInsert1_john_4"},
}).Insert()
if err == nil {
fmt.Println(r.RowsAffected())
} else {
fmt.Println(err)
}
fmt.Println()
}
// 链式批量写入,指定每批次写入的条数
func linkopBatchInsert2() {
fmt.Println("linkopBatchInsert2:")
r, err := db.Table("aa_user").Data(gdb.List{
{"id":25,"name": "linkopBatchInsert2john_1"},
{"id":26,"name": "linkopBatchInsert2john_2"},
{"id":27,"name": "linkopBatchInsert2john_3"},
{"id":28,"name": "linkopBatchInsert2john_4"},
}).Batch(2).Insert()
if err == nil {
fmt.Println(r.RowsAffected())
} else {
fmt.Println(err)
}
fmt.Println()
}
// 链式批量保存
func linkopBatchSave() {
fmt.Println("linkopBatchSave:")
r, err := db.Table("aa_user").Data(gdb.List{
{"id":1, "name": "john_1"},
{"id":2, "name": "john_2"},
{"id":3, "name": "john_3"},
{"id":4, "name": "john_4"},
}).Save()
if err == nil {
fmt.Println(r.RowsAffected())
} else {
fmt.Println(err)
}
fmt.Println()
}
// 事务操作示例1
func transaction1() {
fmt.Println("transaction1:")
if tx, err := db.Begin(); err == nil {
r, err := tx.Insert("aa_user", gdb.Map{
"id" : 30,
"name" : "transaction1",
})
tx.Rollback()
fmt.Println(r, err)
}
fmt.Println()
}
// 事务操作示例2
func transaction2() {
fmt.Println("transaction2:")
if tx, err := db.Begin(); err == nil {
r, err := tx.Table("user_detail").Data(gdb.Map{"id":6, "site": "www.baidu.com哈哈哈*?''\"~!@#$%^&*()"}).Insert()
tx.Commit()
fmt.Println(r, err)
}
fmt.Println()
}
// 主从io复用测试在mysql中使用 show full processlist 查看链接信息
func keepPing() {
fmt.Println("keepPing:")
for i := 0; i < 30; i++ {
fmt.Println("ping...",i)
err := db.PingMaster()
if err != nil {
fmt.Println(err)
return
}
err = db.PingSlave()
if err != nil {
fmt.Println(err)
return
}
time.Sleep(1*time.Second)
}
}
// like语句查询
func likeQuery() {
fmt.Println("likeQuery:")
if r, err := db.Table("aa_user").Where("name like ?", "%john%").Select(); err == nil {
fmt.Println(r)
} else {
fmt.Println(err)
}
}
// mapToStruct
func mapToStruct() {
type User struct {
Id int
Name string
Age int
Addr string
}
fmt.Println("mapToStruct:")
if r, err := db.Table("aa_user").Where("id=?", 1).One(); err == nil {
u := User{}
if err := r.ToStruct(&u); err == nil {
fmt.Println(r)
fmt.Println(u)
} else {
fmt.Println(err)
}
} else {
fmt.Println(err)
}
}
// getQueriedSqls
func getQueriedSqls() {
for k, v := range db.GetQueriedSqls() {
fmt.Println(k, ":")
fmt.Println("Sql :", v.Sql)
fmt.Println("Args :", v.Args)
fmt.Println("Error:", v.Error)
fmt.Println("Func :", v.Func)
}
}
func main() {
db.PingMaster()
db.SetDebug(true)
/*err := create()
if err != nil {
return
}*/
//test1
/*for i := 1; i < 5; i++ {
insert(i)
}*/
//insert(2)
query()
//batchInsert()
//query()
//replace()
//save()
/*update1()
update2()
update3()
*/
/*linkopSelect1()
linkopSelect2()
linkopSelect3()
linkopCount1()
*/
/*linkopUpdate1()
linkopUpdate2()
linkopUpdate3()
linkopUpdate4()
*/
//linkopBatchInsert1()
//linkopBatchInsert2()
//transaction1()
//transaction2()
//
//keepPing()
//likeQuery()
//mapToStruct()
//getQueriedSqls()
}

View File

@ -0,0 +1,13 @@
# MySQL数据库配置
[database]
[[database.default]]
host = "127.0.0.1"
port = "3306"
user = "root"
pass = "12345678"
name = "test"
type = "mysql"
role = "master"
charset = "utf8"
priority = "1"

View File

@ -291,7 +291,7 @@ func linkopSelect3() {
// 链式查询数量1
func linkopCount1() {
fmt.Println("linkopCount1:")
r, err := db.Table("user u").LeftJoin("user_detail ud", "u.uid=ud.uid").Where("u.uid=?", 1).Count()
r, err := db.Table("user u").Fields("uid").LeftJoin("user_detail ud", "u.uid=ud.uid").Where("u.uid=?", 1).Count()
if err == nil {
fmt.Println(r)
} else {
@ -502,7 +502,7 @@ func main() {
//update1()
//update2()
//update3()
//linkopSelect1()
linkopSelect1()
//linkopSelect2()
//linkopSelect3()
//linkopCount1()
@ -511,11 +511,11 @@ func main() {
//linkopUpdate3()
//linkopUpdate4()
//
transaction1()
transaction2()
//transaction1()
//transaction2()
//
//keepPing()
//likeQuery()
mapToStruct()
//mapToStruct()
//getQueriedSqls()
}

View File

@ -0,0 +1,29 @@
package main
import (
"fmt"
"gitee.com/johng/gf/g"
"gitee.com/johng/gf/g/os/gtime"
)
func main() {
db := g.Database()
db.SetDebug(true)
//r, err := db.Table("user").Data("create_time", gtime.Now().String()).Insert()
//if err == nil {
// fmt.Println(r.LastInsertId())
//} else {
// panic(err)
//}
r, err := db.Table("user").Data(g.Map{
"name" : "john",
"create_time" : gtime.Now().String(),
}).Insert()
if err == nil {
fmt.Println(r.LastInsertId())
} else {
panic(err)
}
}

View File

@ -0,0 +1,36 @@
package main
import (
"gitee.com/johng/gf/g/database/gdb"
"time"
)
func main() {
gdb.AddDefaultConfigNode(gdb.ConfigNode {
Host : "127.0.0.1",
Port : "3306",
User : "root",
Pass : "12345678",
Name : "test",
Type : "mysql",
Role : "master",
Charset : "utf8",
MaxIdleConnCount : 10,
MaxOpenConnCount : 10,
MaxConnLifetime : 10,
})
db, err := gdb.New()
if err != nil {
panic(err)
}
// 开启调试模式以便于记录所有执行的SQL
db.SetDebug(true)
for {
for i := 0; i < 10; i++ {
go db.Table("user").All()
}
time.Sleep(time.Second)
}
}

View File

@ -1,16 +1,16 @@
package main
import (
"gitee.com/johng/gf/g/database/gdb"
"fmt"
"gitee.com/johng/gf/g/database/gdb"
)
func main() {
gdb.AddDefaultConfigNode(gdb.ConfigNode {
Host : "127.0.0.1",
Host : "192.168.1.11",
Port : "3306",
User : "root",
Pass : "123456",
Pass : "8692651",
Name : "test",
Type : "mysql",
Role : "master",
@ -23,9 +23,8 @@ func main() {
// 开启调试模式以便于记录所有执行的SQL
db.SetDebug(true)
r, _ := db.Table("user").Where("uid=?", 1).One()
r, _ := db.Table("user").All()
if r != nil {
fmt.Println(r["uid"].Int())
fmt.Println(r["name"].String())
fmt.Println(r.ToList())
}
}

View File

@ -3,7 +3,7 @@ package main
import (
"fmt"
"time"
_ "github.com/mattn/go-oci8"
//_ "github.com/mattn/go-oci8"
"gitee.com/johng/gf/g/database/gdb"
"gitee.com/johng/gf/g"
)

View File

@ -9,7 +9,7 @@ func main() {
s := g.Server()
s.SetIndexFolder(true)
s.BindHandler("/", func(r *ghttp.Request){
r.Response.Writeln("哈喽世界!")
r.Response.Write("Hello World")
})
s.SetPort(8199)
s.Run()

View File

@ -0,0 +1,20 @@
package main
import (
"gitee.com/johng/gf/g"
"gitee.com/johng/gf/g/net/ghttp"
)
func main() {
s := g.Server()
s.BindHandler("/", func(r *ghttp.Request) {
r.Response.Writeln(r.GetParam("name").String())
})
s.BindHookHandlerByMap("/", map[string]ghttp.HandlerFunc {
ghttp.HOOK_BEFORE_SERVE : func(r *ghttp.Request) {
r.SetParam("name", "john")
},
})
s.SetPort(8199)
s.Run()
}

Some files were not shown because too many files have changed in this diff Show More