TL;DR
- インフラの構成をコードで管理したいのでgitlabを使う
- gitlabのレポジトリにPushしたコードは自動でテストをまわしたいのでjenkinsを使う
- サーバ2台使うとリソースもったいたいので1台にまとめてnginxでリバースプロキシする
- gitlabとjenkinsはアクセスするサブディレクトリで分ける
- インスタンスはOpenStack上で動作
これらをセットアップしたときのメモです
構成
Jenkinsのセットアップ
- インストールはここを参照
- サブディレクトリ /jenkins へのアクセスに対応するよう /etc/sysconfig/jenkinsを編集する(通常は http://xxx.xxx.xxx.xxx:8080 がjenkinsのURL)
- JENKINS_ARGS に --prefix=/jenkins を渡す
## Type: string
## Default: ""
## ServiceRestart: jenkins
#
# Pass arbitrary arguments to Jenkins.
# Full option list: java -jar jenkins.war --help
#
JENKINS_ARGS="--prefix=/jenkins"
# service jenkins restart
- /var/log/jenkins/jenkins.log にエラーが出ていないことを確認
WARNING: Could not intialize the host network interface on nullbecause of an error: infra-ci-jenkins.novalocal
: infra-ci-jenkins.novalocal: unknown error
java.net.UnknownHostException: infra-ci-jenkins.novalocal: infra-ci-jenkins.novalocal: unknown error
at java.net.InetAddress.getLocalHost(InetAddress.java:1505)
at javax.jmdns.impl.HostInfo.newHostInfo(HostInfo.java:75)
at javax.jmdns.impl.JmDNSImpl.<init>(JmDNSImpl.java:407)
at javax.jmdns.JmDNS.create(JmDNS.java:60)
at hudson.DNSMultiCast$1.call(DNSMultiCast.java:32)
at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.UnknownHostException: infra-ci-jenkins.novalocal: unknown error
at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928)
at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323)
at java.net.InetAddress.getLocalHost(InetAddress.java:1500)
... 9 more
Gitlabのセットアップ
##################
# GitLab Unicorn #
##################
## Tweak unicorn settings.
# unicorn['worker_timeout'] = 60
# unicorn['worker_processes'] = 2
## Advanced settings
unicorn['listen'] = '127.0.0.1'
unicorn['port'] = 8888
# unicorn['socket'] = '/var/opt/gitlab/gitlab-rails/sockets/gitlab.socket'
# unicorn['pidfile'] = '/opt/gitlab/var/unicorn/unicorn.pid'
# unicorn['tcp_nopush'] = true
# unicorn['backlog_socket'] = 1024
# Make sure somaxconn is equal or higher then backlog_socket
# unicorn['somaxconn'] = 1024
# We do not recommend changing this setting
# unicorn['log_directory'] = "/var/log/gitlab/unicorn"
## Only change these settings if you understand well what they mean
## see https://about.gitlab.com/2015/06/05/how-gitlab-uses-unicorn-and-unicorn-worker-er/
## and https://github.com/kzk/unicorn-worker-killer
# unicorn['worker_memory_limit_min'] = "200*(1024**2)"
# unicorn['worker_memory_limit_max'] = "250*(1024**2)"
<snip>
################
# GitLab Nginx #
################
## see: https://gitlab.com/gitlab-org/omnibus-gitlab/e/629def0a7a26e7c2326566f0758d4a27857b52a3/doc/set tings/nginx.md
# nginx['enable'] = true
# nginx['client_max_body_size'] = '250m'
# nginx['redirect_http_to_https'] = false
# nginx['redirect_http_to_https_port'] = 80
# nginx['ssl_client_certificate'] = "/etc/gitlab/ssl/ca.crt" # Most root CA's are luded by default
# nginx['ssl_certificate'] = "/etc/gitlab/ssl/#{node['fqdn']}.crt"
# nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/#{node['fqdn']}.key"
# nginx['ssl_ciphers'] = "ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256"
# nginx['ssl_prefer_server_ciphers'] = "on"
# nginx['ssl_protocols'] = "TLSv1 TLSv1.1 TLSv1.2" # recommended by https://raymii.orgtutorials/Strong_ SSL_Security_On_nginx.html & https://cipherli.st/
# nginx['ssl_session_cache'] = "builtin:1000 shared:SSL:10m" # recommended in http://nx.org/en/docs/ht tp/ngx_http_ssl_module.html
# nginx['ssl_session_timeout'] = "5m" # default according to http://nginx.org/en/docs/p/ngx_http_ssl_mo dule.html
# nginx['ssl_dhparam'] = nil # Path to dhparams.pem, eg. /etc/gitlab/ssl/dhparams.pem
# nginx['listen_addresses'] = ['*']
# nginx['listen_port'] = nil # override only if you use a reverse proxy: https://lab.com/gitlab-org/omn ibus-gitlab/blob/master/doc/settings/nginx.md#setting-the-nx-listen-port
nginx['listen_port'] = 10080
# nginx['listen_https'] = nil # override only if your reverse proxy internally municates over HTTP: htt ps://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/tings/nginx.md#supporting-proxied-ssl
nginx['listen_https'] = false
# nginx['custom_gitlab_server_config'] = "location ^~ /foo-namespace/bar-project/raw/ deny all;\n}\n"
# nginx['custom_nginx_config'] = "include /etc/nginx/conf.d/example.conf;"
# nginx['proxy_read_timeout'] = 300
# nginx['proxy_connect_timeout'] = 300
- gitlabを再構成する
- gitlabは構成管理にchefが使われている
- 前述のgitlab.rbはそのためのAttributeファイルになっている
# gitlab-ctl reconfigure
gitlab-ctlでは構成されない範囲があるので以下は手動で各設定ファイルを編集していく必要がある
- /opt/gitlab/embedded/service/gitlab-rails/config/application.rb
- config.relative_url_root = "/gitlab" のコメントアウトを外す
# Relative url support
# Uncomment and customize the last line to run in a non-root path
# WARNING: We recommend creating a FQDN to host GitLab in a root path instead of
# Note that following settings need to be changed for this to work.
# 1) In your application.rb file: config.relative_url_root = "/gitlab"
# 2) In your gitlab.yml file: relative_url_root: /gitlab
# 3) In your unicorn.rb: ENV['RAILS_RELATIVE_URL_ROOT'] = "/gitlab"
# 4) In ../gitlab-shell/config.yml: gitlab_url: "http://127.0.0.1/gitlab"
# 5) In lib/support/nginx/gitlab : do not use asset gzipping, remove block g with "location ~ ^ /(assets)/"
#
# To update the path, run: sudo -u git -H bundle exec rake assets:precompile NV=production
#
config.relative_url_root = "/gitlab"
- /var/opt/gitlab/gitlab-rails/etc/gitlab.yml
- relative_url_root: /gitlab のコメントアウトを外す
# WARNING: See config/application.rb under "Relative url support" for the list of
# other files that need to be changed for relative url support
relative_url_root: /gitlab
- /var/opt/gitlab/gitlab-rails/etc/unicorn.rb
- ENV['RAILS_RELATIVE_URL_ROOT'] = "/gitlab" を最終行に追記する
ENV['RAILS_RELATIVE_URL_ROOT'] = "/gitlab"
- /var/opt/gitlab/gitlab-shell/config.yml
gitlab_url: "http://127.0.0.1:8888/gitlab"
# gitlab-ctl restart
- /var/log/gitlab/nginx/gitlab_error.log や /var/log/gitlab/unicorn/unicorn_stderr.logにエラーが出力されていないことを確認
I, [2016-01-12T00:50:57.143568 #26720] INFO -- : Refreshing Gem list
E, [2016-01-12T00:51:11.272678 #26720] ERROR -- : adding listener failed addr=127.0.0.1:8080 (in use)
E, [2016-01-12T00:51:11.272884 #26720] ERROR -- : retrying in 0.5 seconds (4 tries left)
E, [2016-01-12T00:51:11.773372 #26720] ERROR -- : adding listener failed addr=127.0.0.1:8080 (in use)
E, [2016-01-12T00:51:11.774122 #26720] ERROR -- : retrying in 0.5 seconds (3 tries left)
E, [2016-01-12T00:51:12.274524 #26720] ERROR -- : adding listener failed addr=127.0.0.1:8080 (in use)
E, [2016-01-12T00:51:12.274673 #26720] ERROR -- : retrying in 0.5 seconds (2 tries left)
E, [2016-01-12T00:51:12.775115 #26720] ERROR -- : adding listener failed addr=127.0.0.1:8080 (in use)
E, [2016-01-12T00:51:12.775266 #26720] ERROR -- : retrying in 0.5 seconds (1 tries left)
E, [2016-01-12T00:51:13.275751 #26720] ERROR -- : adding listener failed addr=127.0.0.1:8080 (in use)
E, [2016-01-12T00:51:13.275970 #26720] ERROR -- : retrying in 0.5 seconds (0 tries left)
E, [2016-01-12T00:51:13.776510 #26720] ERROR -- : adding listener failed addr=127.0.0.1:8080 (in use)
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/unicorn-4.8.3/lib/unicorn/socket_helper.rb:185:in `bind': Address already in use - bind(2) for 127.0.0.1:8080 (Errno::EADDRINUSE)
from /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/unicorn-4.8.3/lib/unicorn/socket_helper.rb:185:in `new_tcp_server'
from /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/unicorn-4.8.3/lib/unicorn/socket_helper.rb:165:in `bind_listen'
from /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/unicorn-4.8.3/lib/unicorn/http_server.rb:242:in `listen'
from /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/unicorn-4.8.3/lib/unicorn/http_server.rb:809:in `block in bind_new_listeners!'
from /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/unicorn-4.8.3/lib/unicorn/http_server.rb:809:in `each'
from /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/unicorn-4.8.3/lib/unicorn/http_server.rb:809:in `bind_new_listeners!'
from /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/unicorn-4.8.3/lib/unicorn/http_server.rb:138:in `start'
from /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/unicorn-4.8.3/bin/unicorn:126:in `<top (required)>'
from /opt/gitlab/embedded/service/gem/ruby/2.1.0/bin/unicorn:23:in `load'
from /opt/gitlab/embedded/service/gem/ruby/2.1.0/bin/unicorn:23:in `<main>'
nginxのセットアップ
upstream jenkins_server {
server 127.0.0.1:8080 fail_timeout=0;
}
upstream gitlab_server {
server 127.0.0.1:8888 fail_timeout=0;
}
- /etc/nginx.conf/conf.d/default.confを以下のように編集する(ここではdefault.confはファイル名を変更し、basic.confとしている)
server {
listen 80;
server_name localhost;
#charset koi8-r;
#access_log /var/log/nginx/log/host.access.log main;
location / {
root /usr/share/nginx/html;
index index.html index.htm;
}
location = /50x.html {
root /usr/share/nginx/html;
}
# Jenkins
location ~ /jenkins {
proxy_read_timeout 300;
proxy_connect_timeout 300;
proxy_redirect off;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $http_host;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://jenkins_server;
access_log /var/log/nginx/jenkins_access.log;
error_log /var/log/nginx/jenkins_error.log;
}
# gitlab
location ~ /gitlab {
proxy_read_timeout 300;
proxy_connect_timeout 300;
proxy_redirect off;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $http_host;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://gitlab_server;
access_log /var/log/nginx/gitlab_access.log;
error_log /var/log/nginx/gitlab_error.log;
}
}
# chkconfig nginx on
# service nginx restart
- /var/log/nginx/error.log、/var/log/nginx/jenkins_error.log、 /var/log/nginx/gitlab_error.logにエラーが出力されていないことを確認
[error] 1226#0: *225 connect() failed (111: Connection refused) while connecting to upstream, client: 172.24.4.254, server: localhost, request: "GET /gitlab HTTP/1.1", upstream: "http://127.0.0.1:8888/gitlab", host: "10.0.1.63:13580"
接続してみる
http://xxx.xxx.xxx.xxx/jenkins と http://xxx.xxx.xxx.xxx/gitlab にアクセスして、それぞれのサービスのトップ画面が表示されればOK
done.