tunasync是清华大学TUNA镜像源目前使用的镜像方案，github地址：https://github.com/tuna/tunasync
前端页面代码来自：https://github.com/weyo/mirrors
tunasync服务启动停止脚本：https://github.com/whsir/tunasync-bin
CentOS源列表：https://www.centos.org/download/mirrors/
EPEL源列表：https://admin.fedoraproject.org/mirrormanager/mirrors/EPEL
当前以CentOS7系统为例，搭建一个基于内网的镜像仓库，建议硬盘至少2T以上，同步速度与网络带宽、磁盘读写速度有关！
1、安装依赖
|
1 |
yum install rsync wget |
2、下载tunasync
|
1 2 3 4 |
wget https://github.com/tuna/tunasync/releases/download/v0.3.7/tunasync-linux-bin.tar.gz tar xf tunasync-linux-bin.tar.gz mv tunasync /usr/bin/ mv tunasynctl /usr/bin/ |
3、创建仓库目录
|
1 |
mkdir -p /data/mirrors |
4、创建日志目录
|
1 |
mkdir -p /data/logs/tunasync |
5、创建tunasync配置文件
|
1 |
mkdir /etc/tunasync |
6、创建manager.conf配置文件
|
1 |
vi /etc/tunasync/manager.conf |
|
1 2 3 4 5 6 7 8 9 10 11 12 |
debug = false [server] addr = "127.0.0.1" port = 12345 ssl_cert = "" ssl_key = "" [files] db_type = "bolt" db_file = "/etc/tunasync/manager.db" ca_cert = "" |
manager部分参数说明：
addr：manager服务监听地址
port：manager服务监听端口
ssl_cert和ssl_key：证书配置
db_file：数据库文件位置
7、创建worker.conf配置文件
|
1 |
vi /etc/tunasync/worker.conf |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 |
[global] name = "test_worker" log_dir = "/data/logs/tunasync/{{.Name}}" mirror_dir = "/data/mirrors" concurrent = 10 interval = 240 [manager] api_base = "http://127.0.0.1:12345" token = "" ca_cert = "" [cgroup] enable = false base_path = "/sys/fs/cgroup" group = "tunasync" [server] hostname = "localhost" listen_addr = "127.0.0.1" listen_port = 6000 ssl_cert = "" ssl_key = "" [[mirrors]] name = "centos" provider = "rsync" upstream = "rsync://mirrors.tuna.tsinghua.edu.cn/centos/" use_ipv6 = false [[mirrors]] name = "epel" provider = "rsync" upstream = "rsync://mirrors.tuna.tsinghua.edu.cn/epel/" use_ipv6 = false |
worker部分参数说明：
log_dir：tunasync日志路径
mirror_dir：镜像的路径
concurrent：线程数
interval：同步周期，单位分钟
api_base：manager地址
name：镜像名称
upstream：upstream同步地址
8、下载tunasync启动停止脚本
|
1 2 3 4 |
wget -P /etc/init.d/ https://raw.githubusercontent.com/whsir/tunasync-bin/master/tunasync-manager wget -P /etc/init.d/ https://raw.githubusercontent.com/whsir/tunasync-bin/master/tunasync-worker chmod +x /etc/init.d/tunasync-manager chmod +x /etc/init.d/tunasync-worker |
9、启动服务
|
1 2 |
/etc/init.d/tunasync-manager start /etc/init.d/tunasync-worker start |
10、查看同步状态
此处12345为manager的端口
|
1 |
tunasynctl list -p 12345 --all |
11、添加计划任务
|
1 |
*/1 * * * * wget -q http://127.0.0.1:12345/jobs -O /data/mirrors/jobs.json |
12、配置前端web
|
1 2 |
rpm -ivh http://mirrors.wlnmp.com/centos/wlnmp-release-centos.noarch.rpm yum install wnginx |
13、配置nginx
|
1 |
vi /usr/local/nginx/conf/vhost/demo.conf |
修改后的内容如下
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 |
server { listen 80 default_server; server_name _; root /data/mirrors; location ~ ^/(centos|epel) { index index.html index.htm; autoindex on; autoindex_exact_size off; autoindex_localtime on; } error_page 500 502 503 504 /50x.html; #include enable-php56.conf; #include enable-php70.conf; #include enable-php71.conf; #include enable-php72.conf; #include enable-php73.conf; #include enable-php74.conf; location ~ .*\.(gif|jpg|jpeg|png|bmp|swf)$ { expires 30d; } location ~ .*\.(js|css)?$ { expires 12h; } access_log /data/logs/nginx/demo_access.log; error_log /data/logs/nginx/demo_error.log; } |
14、设置前端web
|
1 2 |
cd /data/mirrors wget https://down.whsir.com/downloads/mirrors_web.tar.gz |
重启nginx
|
1 |
/etc/init.d/nginx restart |
15、访问测试
访问URL即可看到以下内容，由于这两个源很大，同步时间会非常长
附：我这里再举个例子，基于docker-ce镜像同步
镜像同步脚本：https://github.com/tuna/tunasync-scripts
1、创建镜像同步脚本目录
|
1 2 |
mkdir /etc/tunasync/scripts/ cd /etc/tunasync/scripts/ |
2、下载docker-ce同步脚本
|
1 2 |
wget https://raw.githubusercontent.com/tuna/tunasync-scripts/master/docker-ce.py chmod +x docker-ce.py |
3、在worker中新增docker-ce同步配置
|
1 |
vi /etc/tunasync/worker.conf |
|
1 2 3 4 5 6 |
[[mirrors]] name = "docker-ce" provider = "command" upstream = "https://download.docker.com/" command = "/etc/tunasync/scripts/docker-ce.py" docker_image = "tunathu/tunasync-scripts:latest" |
4、安装docker-ce同步时所需依赖
|
1 2 |
yum install python3 python-pip pip3 install requests pyquery |
5、启动服务
|
1 2 |
/etc/init.d/tunasync-manager start /etc/init.d/tunasync-worker start |
6、修改nginx
在location中继续增加一个docker-ce
|
1 |
vi /usr/local/nginx/conf/vhost/demo.conf |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 |
server { listen 80 default_server; server_name _; root /data/mirrors; location ~ ^/(centos|epel|docker-ce) { index index.html index.htm; autoindex on; autoindex_exact_size off; autoindex_localtime on; } error_page 500 502 503 504 /50x.html; #include enable-php56.conf; #include enable-php70.conf; #include enable-php71.conf; #include enable-php72.conf; #include enable-php73.conf; #include enable-php74.conf; location ~ .*\.(gif|jpg|jpeg|png|bmp|swf)$ { expires 30d; } location ~ .*\.(js|css)?$ { expires 12h; } access_log /data/logs/nginx/demo_access.log; error_log /data/logs/nginx/demo_error.log; } |
7、重启nginx
|
1 |
/etc/init.d/nginx restart |
访问URL即可看到以下内容
前端web都是基于html写的，如果继续添加其他的源，可以自行更改
写在最后
手动同步方式
|
1 |
rsync -aHvh --no-o --no-g --stats --exclude .~tmp~/ --delete --delete-after --delay-updates --safe-links --timeout=120 --contimeout=120 rsync://mirrors.tuna.tsinghua.edu.cn/epel/ /data/mirrors/epel |
|
1 |
rsync -aHvh --no-o --no-g --stats --exclude .~tmp~/ --delete --delete-after --delay-updates --safe-links --timeout=120 --contimeout=120 rsync://mirrors.tuna.tsinghua.edu.cn/centos/ /data/mirrors/centos |
|
1 |
python3 /etc/tunasync/scripts/docker-ce.py --working-dir /data/mirrors/docker-ce/ |
docker-ce同步不建议使用上面python脚本方式，也可以通过rsync增量方式来同步清华大学站的docker-ce
|
1 |
rsync -aHvh --no-o --no-g --stats --exclude .~tmp~/ --delete --delete-after --delay-updates --safe-links --timeout=120 --contimeout=120 rsync://mirrors.tuna.tsinghua.edu.cn/docker-ce/ /data/mirrors/docker-ce |
PS：如果同步后发现清华大学的centos源有个权限的报错
rsync: readlink_stat("7.7.1908/isos/x86_64/.CentOS-7-x86_64-Everything-1908.iso.RjFDl5" (in centos)) failed: Permission denied (13)
可以考虑将centos源更换香港这个源：rsync://mirror.hostlink.com.hk/centos/
原文链接：通过tunasync搭建镜像源仓库，转载请注明来源！
