TL;DR
- インフラの構成をコードで管理したいのでgitlabを使う
- gitlabのレポジトリにPushしたコードは自動でテストをまわしたいのでjenkinsを使う
- サーバ2台使うとリソースもったいたいので1台にまとめてnginxでリバースプロキシする
- gitlabとjenkinsはアクセスするサブディレクトリで分ける
- インスタンスはOpenStack上で動作
これらをセットアップしたときのメモです
構成
- OpenStackのインスタンスで構築
- OSはCentOS6.5
- http://xxx.xxx.xxx.xxx/gitlab へアクセスした場合はgitlabへ
- http://xxx.xxx.xxx.xxx/jenkins へアクセスした場合はjenkinsへ
Jenkinsのセットアップ
- インストールはここを参照
- サブディレクトリ /jenkins へのアクセスに対応するよう /etc/sysconfig/jenkinsを編集する(通常は http://xxx.xxx.xxx.xxx:8080 がjenkinsのURL)
- JENKINS_ARGS に --prefix=/jenkins を渡す
## Type: string ## Default: "" ## ServiceRestart: jenkins # # Pass arbitrary arguments to Jenkins. # Full option list: java -jar jenkins.war --help # JENKINS_ARGS="--prefix=/jenkins"
- jenkinsを再起動する
# service jenkins restart
- /var/log/jenkins/jenkins.log にエラーが出ていないことを確認
WARNING: Could not intialize the host network interface on nullbecause of an error: infra-ci-jenkins.novalocal : infra-ci-jenkins.novalocal: unknown error java.net.UnknownHostException: infra-ci-jenkins.novalocal: infra-ci-jenkins.novalocal: unknown error at java.net.InetAddress.getLocalHost(InetAddress.java:1505) at javax.jmdns.impl.HostInfo.newHostInfo(HostInfo.java:75) at javax.jmdns.impl.JmDNSImpl.<init>(JmDNSImpl.java:407) at javax.jmdns.JmDNS.create(JmDNS.java:60) at hudson.DNSMultiCast$1.call(DNSMultiCast.java:32) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.net.UnknownHostException: infra-ci-jenkins.novalocal: unknown error at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method) at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928) at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323) at java.net.InetAddress.getLocalHost(InetAddress.java:1500) ... 9 more
Gitlabのセットアップ
インストールはここを参照
/etc/gitlab/gitlab.rb を編集する
- gitlabは、Rackサーバのunicornで動作しており、内部で同梱のnginxがリバースプロキシをしている。そのため、以下の箇所でgitlab同梱のunicornとnginxの設定を行っておく必要がある。
- unicorn['listen'] を 127.0.0.1 にする
- unicorn['port'] を 8888 にする(デフォルトの8080はJenkinsで使用している為)
- nginx['listen_port'] を 10080 にする(デフォルトの80はGitlabとJenkinsをリバースプロキシするフロントのnginxで使用するため)
- nginx['listen_https']を false にする(もしSSLにする場合はフロントのnginxで対応すれば良い)
################## # GitLab Unicorn # ################## ## Tweak unicorn settings. # unicorn['worker_timeout'] = 60 # unicorn['worker_processes'] = 2 ## Advanced settings unicorn['listen'] = '127.0.0.1' unicorn['port'] = 8888 # unicorn['socket'] = '/var/opt/gitlab/gitlab-rails/sockets/gitlab.socket' # unicorn['pidfile'] = '/opt/gitlab/var/unicorn/unicorn.pid' # unicorn['tcp_nopush'] = true # unicorn['backlog_socket'] = 1024 # Make sure somaxconn is equal or higher then backlog_socket # unicorn['somaxconn'] = 1024 # We do not recommend changing this setting # unicorn['log_directory'] = "/var/log/gitlab/unicorn" ## Only change these settings if you understand well what they mean ## see https://about.gitlab.com/2015/06/05/how-gitlab-uses-unicorn-and-unicorn-worker-er/ ## and https://github.com/kzk/unicorn-worker-killer # unicorn['worker_memory_limit_min'] = "200*(1024**2)" # unicorn['worker_memory_limit_max'] = "250*(1024**2)" <snip> ################ # GitLab Nginx # ################ ## see: https://gitlab.com/gitlab-org/omnibus-gitlab/e/629def0a7a26e7c2326566f0758d4a27857b52a3/doc/set tings/nginx.md # nginx['enable'] = true # nginx['client_max_body_size'] = '250m' # nginx['redirect_http_to_https'] = false # nginx['redirect_http_to_https_port'] = 80 # nginx['ssl_client_certificate'] = "/etc/gitlab/ssl/ca.crt" # Most root CA's are luded by default # nginx['ssl_certificate'] = "/etc/gitlab/ssl/#{node['fqdn']}.crt" # nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/#{node['fqdn']}.key" # nginx['ssl_ciphers'] = "ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256" # nginx['ssl_prefer_server_ciphers'] = "on" # nginx['ssl_protocols'] = "TLSv1 TLSv1.1 TLSv1.2" # recommended by https://raymii.orgtutorials/Strong_ SSL_Security_On_nginx.html & https://cipherli.st/ # nginx['ssl_session_cache'] = "builtin:1000 shared:SSL:10m" # recommended in http://nx.org/en/docs/ht tp/ngx_http_ssl_module.html # nginx['ssl_session_timeout'] = "5m" # default according to http://nginx.org/en/docs/p/ngx_http_ssl_mo dule.html # nginx['ssl_dhparam'] = nil # Path to dhparams.pem, eg. /etc/gitlab/ssl/dhparams.pem # nginx['listen_addresses'] = ['*'] # nginx['listen_port'] = nil # override only if you use a reverse proxy: https://lab.com/gitlab-org/omn ibus-gitlab/blob/master/doc/settings/nginx.md#setting-the-nx-listen-port nginx['listen_port'] = 10080 # nginx['listen_https'] = nil # override only if your reverse proxy internally municates over HTTP: htt ps://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/tings/nginx.md#supporting-proxied-ssl nginx['listen_https'] = false # nginx['custom_gitlab_server_config'] = "location ^~ /foo-namespace/bar-project/raw/ deny all;\n}\n" # nginx['custom_nginx_config'] = "include /etc/nginx/conf.d/example.conf;" # nginx['proxy_read_timeout'] = 300 # nginx['proxy_connect_timeout'] = 300
- gitlabを再構成する
- gitlabは構成管理にchefが使われている
- 前述のgitlab.rbはそのためのAttributeファイルになっている
# gitlab-ctl reconfigure
gitlab-ctlでは構成されない範囲があるので以下は手動で各設定ファイルを編集していく必要がある
- /opt/gitlab/embedded/service/gitlab-rails/config/application.rb
- config.relative_url_root = "/gitlab" のコメントアウトを外す
# Relative url support # Uncomment and customize the last line to run in a non-root path # WARNING: We recommend creating a FQDN to host GitLab in a root path instead of # Note that following settings need to be changed for this to work. # 1) In your application.rb file: config.relative_url_root = "/gitlab" # 2) In your gitlab.yml file: relative_url_root: /gitlab # 3) In your unicorn.rb: ENV['RAILS_RELATIVE_URL_ROOT'] = "/gitlab" # 4) In ../gitlab-shell/config.yml: gitlab_url: "http://127.0.0.1/gitlab" # 5) In lib/support/nginx/gitlab : do not use asset gzipping, remove block g with "location ~ ^ /(assets)/" # # To update the path, run: sudo -u git -H bundle exec rake assets:precompile NV=production # config.relative_url_root = "/gitlab"
# WARNING: See config/application.rb under "Relative url support" for the list of # other files that need to be changed for relative url support relative_url_root: /gitlab
ENV['RAILS_RELATIVE_URL_ROOT'] = "/gitlab"
- /var/opt/gitlab/gitlab-shell/config.yml
- gitlab_url: "http://127.0.0.1:8888" を gitlab_url: "http://127.0.0.1:8888/gitlab" に変更する
gitlab_url: "http://127.0.0.1:8888/gitlab"
- gitlabを再起動する
# gitlab-ctl restart
- /var/log/gitlab/nginx/gitlab_error.log や /var/log/gitlab/unicorn/unicorn_stderr.logにエラーが出力されていないことを確認
I, [2016-01-12T00:50:57.143568 #26720] INFO -- : Refreshing Gem list E, [2016-01-12T00:51:11.272678 #26720] ERROR -- : adding listener failed addr=127.0.0.1:8080 (in use) E, [2016-01-12T00:51:11.272884 #26720] ERROR -- : retrying in 0.5 seconds (4 tries left) E, [2016-01-12T00:51:11.773372 #26720] ERROR -- : adding listener failed addr=127.0.0.1:8080 (in use) E, [2016-01-12T00:51:11.774122 #26720] ERROR -- : retrying in 0.5 seconds (3 tries left) E, [2016-01-12T00:51:12.274524 #26720] ERROR -- : adding listener failed addr=127.0.0.1:8080 (in use) E, [2016-01-12T00:51:12.274673 #26720] ERROR -- : retrying in 0.5 seconds (2 tries left) E, [2016-01-12T00:51:12.775115 #26720] ERROR -- : adding listener failed addr=127.0.0.1:8080 (in use) E, [2016-01-12T00:51:12.775266 #26720] ERROR -- : retrying in 0.5 seconds (1 tries left) E, [2016-01-12T00:51:13.275751 #26720] ERROR -- : adding listener failed addr=127.0.0.1:8080 (in use) E, [2016-01-12T00:51:13.275970 #26720] ERROR -- : retrying in 0.5 seconds (0 tries left) E, [2016-01-12T00:51:13.776510 #26720] ERROR -- : adding listener failed addr=127.0.0.1:8080 (in use) /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/unicorn-4.8.3/lib/unicorn/socket_helper.rb:185:in `bind': Address already in use - bind(2) for 127.0.0.1:8080 (Errno::EADDRINUSE) from /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/unicorn-4.8.3/lib/unicorn/socket_helper.rb:185:in `new_tcp_server' from /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/unicorn-4.8.3/lib/unicorn/socket_helper.rb:165:in `bind_listen' from /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/unicorn-4.8.3/lib/unicorn/http_server.rb:242:in `listen' from /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/unicorn-4.8.3/lib/unicorn/http_server.rb:809:in `block in bind_new_listeners!' from /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/unicorn-4.8.3/lib/unicorn/http_server.rb:809:in `each' from /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/unicorn-4.8.3/lib/unicorn/http_server.rb:809:in `bind_new_listeners!' from /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/unicorn-4.8.3/lib/unicorn/http_server.rb:138:in `start' from /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/unicorn-4.8.3/bin/unicorn:126:in `<top (required)>' from /opt/gitlab/embedded/service/gem/ruby/2.1.0/bin/unicorn:23:in `load' from /opt/gitlab/embedded/service/gem/ruby/2.1.0/bin/unicorn:23:in `<main>'
nginxのセットアップ
nginxのインストールはここを参照
/etc/nginx/conf.d/ に以下のファイルを作成する
- jenkins.conf
- gitlab.conf
upstream jenkins_server { server 127.0.0.1:8080 fail_timeout=0; }
upstream gitlab_server { server 127.0.0.1:8888 fail_timeout=0; }
- /etc/nginx.conf/conf.d/default.confを以下のように編集する(ここではdefault.confはファイル名を変更し、basic.confとしている)
server { listen 80; server_name localhost; #charset koi8-r; #access_log /var/log/nginx/log/host.access.log main; location / { root /usr/share/nginx/html; index index.html index.htm; } location = /50x.html { root /usr/share/nginx/html; } # Jenkins location ~ /jenkins { proxy_read_timeout 300; proxy_connect_timeout 300; proxy_redirect off; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header Host $http_host; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_pass http://jenkins_server; access_log /var/log/nginx/jenkins_access.log; error_log /var/log/nginx/jenkins_error.log; } # gitlab location ~ /gitlab { proxy_read_timeout 300; proxy_connect_timeout 300; proxy_redirect off; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header Host $http_host; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_pass http://gitlab_server; access_log /var/log/nginx/gitlab_access.log; error_log /var/log/nginx/gitlab_error.log; } }
/etc/nginx/conf.d/example_ssl.conf ファイルを削除する
nginxを再起動する
# chkconfig nginx on # service nginx restart
- /var/log/nginx/error.log、/var/log/nginx/jenkins_error.log、 /var/log/nginx/gitlab_error.logにエラーが出力されていないことを確認
[error] 1226#0: *225 connect() failed (111: Connection refused) while connecting to upstream, client: 172.24.4.254, server: localhost, request: "GET /gitlab HTTP/1.1", upstream: "http://127.0.0.1:8888/gitlab", host: "10.0.1.63:13580"
接続してみる
http://xxx.xxx.xxx.xxx/jenkins と http://xxx.xxx.xxx.xxx/gitlab にアクセスして、それぞれのサービスのトップ画面が表示されればOK
- jenkins
- gitlab
done.