Docker - Revision history

Michael.mast: /* Poor Man's Cluster */

2019-11-06T16:08:46Z

Poor Man's Cluster

Michael.mast: /* Poor Man's Cluster */

2019-11-06T15:29:42Z

Poor Man's Cluster

Michael.mast at 13:49, 6 November 2019

2019-11-06T13:49:14Z

Michael.mast: /* SELinux */

2019-07-09T00:33:15Z

SELinux

Michael.mast: /* SELinux */

2019-07-09T00:32:30Z

SELinux

Michael.mast at 13:41, 6 April 2019

2019-04-06T13:41:47Z

Michael.mast at 17:49, 4 March 2019

2019-03-04T17:49:20Z

Michael.mast: Created page with "==Networking== ===IP Forwarding=== This is required to allow the containers to route out of the local docker network.https://www.centos.org/docs/5/html/Virtual_Server_Adm..."

2018-11-19T14:31:34Z

Created page with "==Networking== ===IP Forwarding=== This is required to allow the containers to route out of the local docker network.<ref>https://www.centos.org/docs/5/html/Virtual_Server_Adm..."

New page

==Networking==
===IP Forwarding===
This is required to allow the containers to route out of the local docker network.<ref>https://www.centos.org/docs/5/html/Virtual_Server_Administration/s1-lvs-forwarding-VSA.html</ref>
<pre>
echo 'net.ipv4.ip_forward = 1' >> /etc/sysctl.conf
systemctl restart network
</pre>

@@ Line 38: / Line 38: @@
 *All saves images are stored in /mnt/docker-efs/containers
 <br>
-Either host can build the containers if load is not too high. Let's build on host-1
+Either host can build the containers if load is not too high. In this example I am building on host-1. Also note that I build a custom apache/php container that is used as the base for many others. The script creates this first then creates the others.
 <pre>
 #!/bin/bash

← Older revision		Revision as of 15:29, 6 November 2019
Line 24:		Line 24:
	==Clustering==		==Clustering==
	===Poor Man's Cluster===		===Poor Man's Cluster===
		+	I started to use the base docker daemon because containers were easier to work with. I was able to run many services off a single host without conflicts or complicated config files. However, now I needed to make the service run with some redundancy. Maybe a container crashes or the host is down. There are many good ways to handle this, but for stupid reasons I wanted to handle this with shell scripts. At the moment I do not have any health checks implemented, as long as the second host is online I should be ok. However, health checks are on the 'ol to-do list.
		+	<br>
		+	<br>
		+	*The environment is AWS.
		+	*Both hosts are running on Amazon Linux 2
		+	*VPC in Ohio Region
		+	*Different subnets/AZs.
		+	*Share an EFS mount so the files are the same (These are primarily web containers. This allows the files to be modified on the fly)
		+	*Share an RDS MySQL Serverless database.
		+	*All the files to build the containers are in /mnt/docker-efs/build (This uses sub directories for each container)
		+	*All directories mounted into containers are in /mnt/docker-efs/sitefiles
		+	*All start scripts for the containers are in /mnt/docker-efs/start_scripts
		+	*All saves images are stored in /mnt/docker-efs/containers
		+	<br>
		+	Either host can build the containers if load is not too high. Let's build on host-1
		+	<pre>
		+	#!/bin/bash
		+	##Build all containers, mark them as latest.
		+	buildbase='/mnt/docker-efs/'
		+	containerbase="$buildbase/containers"
		+	today=`date +%F`
		+	list=$(ls $buildbase/build)
		+
		+	function _check() {
		+	[ "$?" -ne '0' ] && echo 'Script failed' && exit
		+	}
		+
		+	function _save() {
		+	docker save "$1" > "$containerbase"/"$1".tar
		+	_check
		+	}
		+
		+	function _build() {
		+	local buildsources="build/$1"
		+	docker build -t "$1":${today} -f "$buildbase"/"$buildsources"/[d..D]ockerfile "$buildbase"/"$buildsources"/ --no-cache
		+	_check
		+	_save "$1":${today}
		+	docker tag "$1":${today} "$1":latest
		+	}
		+
		+	[ -z "$1" ] && echo -e 'Options : \n\nlist\nbuild\nbackup' && exit
		+	[ "$1" == 'list' ] && echo "$list" && exit
		+	[ "$1" == 'build' ] && docker system prune -a -f && _build localphp && for i in $list; do [ "$i" != "localphp" ] && echo "Building $i" && _build "$i"; done && exit
		+	[ "$1" == 'backup' ] && echo "place holder for saves" && exit
		+
		+	echo "You should not see this, something went wrong."
		+	</pre>
		+	After the containers have been built, they need to be imported into the other host. On host-2 run the import script.
		+	<pre>
		+	#!/bin/bash
		+	source='/mnt/docker-efs/containers'
		+
		+	list=`ls -1t /mnt/docker-efs/containers/`
		+	[ ! -z "$list" ] && date=`echo $list \| awk '{print $1}' \| grep -oE 20[0-9]\{2\}-[0-9]\{2\}-[0-9]\{2\}`
		+	[ ! -z "$date" ] && find "$source" -name "$date" \| for i in `cat -` ; do
		+	name=`echo $i \| grep -oE [a-z_]+: \| sed 's/://'`
		+	[ ! -z $name ] && docker load --input "$i"
		+	[ "$?" == "0" ] && docker tag "$name":"$date" "$name":latest
		+	done
		+	/mnt/docker-efs/reload.sh
		+	docker system prune -a -f
		+	</pre>
		+	The above script will also run the reload.sh script. This tells the host to run execute all the start scripts. The first command in each start script is to kill the associated running container. This will ensure fresh containers are loaded.
		+	<pre>
		+	#!/bin/bash
		+	##Delete then run all containers.
		+	source='/mnt/docker-efs/start_scripts'
		+	ls "$source"/ \| for i in `cat -`; do
		+	[ ! -z "$i" ] && "$source"/"$i"
		+	done
		+	</pre>
		+	Now that I have the same containers running on two hosts, and the duplicate containers are sharing the same database and files, I can use a standard web proxy to load balance between the hosts. Other services, such as RADIUS, can use round-robin DNS or native load balancing from the connecting client.

← Older revision		Revision as of 13:49, 6 November 2019
Line 22:		Line 22:
	docker run --name steamcache --restart=always -d -v /raid5/archive/steamcache:/data/cache:z -v /raid5/archive/steamcache/logs:/data/logs:z --network=steam -p 192.168.11.200:80:80 steamcache/monolithic:latest		docker run --name steamcache --restart=always -d -v /raid5/archive/steamcache:/data/cache:z -v /raid5/archive/steamcache/logs:/data/logs:z --network=steam -p 192.168.11.200:80:80 steamcache/monolithic:latest
	</pre>		</pre>
		+	==Clustering==
		+	===Poor Man's Cluster===

← Older revision		Revision as of 00:33, 9 July 2019
Line 19:		Line 19:
	</pre>		</pre>
	Or even better, just use the 'z' flag for mounting volumes.<ref>http://jaormx.github.io/2018/selinux-and-docker-notes/</ref>		Or even better, just use the 'z' flag for mounting volumes.<ref>http://jaormx.github.io/2018/selinux-and-docker-notes/</ref>
		+	<pre>
		+	docker run --name steamcache --restart=always -d -v /raid5/archive/steamcache:/data/cache:z -v /raid5/archive/steamcache/logs:/data/logs:z --network=steam -p 192.168.11.200:80:80 steamcache/monolithic:latest
		+	</pre>

← Older revision		Revision as of 00:32, 9 July 2019
Line 18:		Line 18:
	chcon -Rt svirt_sandbox_file_t /directory/to/be/mounted		chcon -Rt svirt_sandbox_file_t /directory/to/be/mounted
	</pre>		</pre>
		+	Or even better, just use the 'z' flag for mounting volumes.<ref>http://jaormx.github.io/2018/selinux-and-docker-notes/</ref>

@@ Line 12: / Line 12: @@
 echo 'net.ipv4.ip_forward = 1' >> /etc/sysctl.conf
 systemctl restart network
 </pre>

← Older revision		Revision as of 17:49, 4 March 2019
Line 1:		Line 1:
		+	==Building Containers==
		+	*Build without working with cached data.
		+	<pre>
		+	#!/bin/bash
		+	docker build -t localphp:`date +%m%d%Y` -f Dockerfile --no-cache .
		+	docker build -t localphp -f Dockerfile .
		+	</pre>
	==Networking==		==Networking==
	===IP Forwarding===		===IP Forwarding===