golang rune


//=== https://golang.org/ref/spec#Rune_literals
Rune literals
"""...
A rune literal represents a rune constant, an integer value identifying a Unicode code point.
rune_lit = "'" ( unicode_value | byte_value ) "'" .
unicode_value = unicode_char | little_u_value | big_u_value | escaped_char .
byte_value = octal_byte_value | hex_byte_value .
octal_byte_value = `\` octal_digit octal_digit octal_digit .
hex_byte_value = `\` "x" hex_digit hex_digit .
little_u_value = `\` "u" hex_digit hex_digit hex_digit hex_digit .
big_u_value = `\` "U" hex_digit hex_digit hex_digit hex_digit
hex_digit hex_digit hex_digit hex_digit .
escaped_char = `\` ( "a" | "b" | "f" | "n" | "r" | "t" | "v" | `\` | "'" | `"` ) .


..."""


golang "..." and '…'


//=== https://golang.org/ref/spec#Passing_arguments_to_..._parameters

* "..." is different from '…'
* The character … (as opposed to the three characters ...) is not a token of the Go language

" ... "
The final parameter in a function signature may have a type prefixed with ... ,
a function with such a parameter is called variadic and may be
invoked with zero or more arguments for that parameter.

func(prefix string, values ...int)
func(a, b int, z float64, opt ...interface{}) (success bool)



'…'

[]T{x1, x2, … xn}
decimal_lit = ( "1" … "9" ) { decimal_digit } .









golang map type


https://blog.golang.org/go-maps-in-action

map === dictionary
mapping of key-value pairs

var m0 = map[string]int{
"tsm": 2330,
"umc": 2303,
"cmc": 2323,
"csc": 2002,
}


var m1 map[string]int //m1 is nil
var m2 = map[string]int{} //m2 points to a map with default initialization (zero values)
var m3 = make(map[string]int) // m3 also points to a map with default initialization


"""...
map[KeyType]ValueType

var m map[string]int

...
Map types are reference type
... the value of m above is nil; it doesn't point to an initialized map.
... A nil map behaves like an empty map when reading, but attempts to write to a nil map will cause a runtime panic; ...
... To initialize a map, use the built in make function:
m = make(map[string]int)


...
i, ok := m["route"]
the first value (i) is assigned the value stored under the key "route".
If that key doesn't exist, i is the value type's zero value (0).

The second value (ok) is a bool that is true if the key exists in the map, and false if not.


To test for a key without retrieving the value, use an underscore in place of the first value:
_, ok := m["route"]


To iterate over the contents of a map, use the range keyword:

for key, value := range m {
fmt.Println("Key:", key, "Value:", value)
}


..."""



[ex] swarm/filter.go
type Filter interface {
// Return a subset of nodes that were accepted by the filtering policy.
Filter(*dockerclient.ContainerConfig, []cluster.Node) ([]cluster.Node, error)
}

var (
filters map[string]Filter
ErrNotSupported = errors.New("filter not supported")
)








golang struct and interface


http://www.golang-book.com/9/index.htm

"""...
type Circle struct {
x float64
y float64
r float64
}

func (c *Circle) area() float64 {
return math.Pi * c.r*c.r
}

[member function for 'class' Circle]
... In between the keyword func and the name of the function we've added a “receiver”.


c := new(Circle)
c := Circle{x: 0, y: 0, r: 5}
c := Circle{0, 0, 5}


...
type Rectangle struct {
x1, y1, x2, y2 float64
}
func (r *Rectangle) area() float64 {
l := distance(r.x1, r.y1, r.x1, r.y2)
w := distance(r.x1, r.y1, r.x2, r.y1)
return l * w
}



... able to name the Rectangle's area method the same thing as the Circle's area method. This was no accident

Go has a way of making these accidental similarities explicit through a type known as an Interface. Here is an example of a Shape interface:

type Shape interface {
area() float64
}


..."""



[anonymous field / embedded type ]

type Person struct {
Name string
}
func (p *Person) Talk() {
fmt.Println("Hi, my name is", p.Name)
}

* Android has a Person
type Android struct {
Pson Person
Model string
}
... we would rather say an Android is a Person, rather than an Android has a Person.

type Android struct {
Person //unamed field of type Person
Model string
}

-->
a := new(Android)
a.Talk() //skip the Person field




golang class and constructor?



//=== class ?
http://www.golangpatterns.info/object-oriented/classes

type Integer int;
func (i *Integer) String() string {
return strconv.itoa(i)
}

--> Integer is a 'class'[type] with member method String()


//=== struct

https://golang.org/ref/spec#Struct_types

"""...
A struct is a sequence of named elements, called fields, each of which has a name and a type

// An empty struct.
struct {}

// A struct with 6 fields.
struct {
x, y int
u float32
_ float32 // padding
A *[]int
F func()
}
A field declared with a type but no explicit field name is an anonymous field
..."""


//=== address operators : & and *
https://golang.org/ref/spec#Operators

& :
For an operand x of type T, the address operation &x generates a pointer of type *T to x.

* :
For an operand x of pointer type *T, the pointer indirection *x denotes the variable of type T pointed to by x





//=== constructor?
http://ww.golangpatterns.info/object-oriented/constructors

Go has no 'real' constructors
--> but 'constructor-like' factory function


"""...
package matrix
function NewMatrix(rows, cols int) *matrix {
m := new(matrix)
m.rows = rows
m.cols = cols
m.elems = make([]float, rows*cols)
return m
}
To prevent users from instantiating uninitialized objects,
the struct can be made private.

package main
import "matrix"
wrong := new(matrix.matrix) // will NOT compile (matrix is private)
right := matrix.NewMatrix(2,3) // ONLY way to instantiate a matrix


...
matrix := NewMatrix(10, 10)
pair := &Pair{"one", 1}

...
function NewMatrix(rows, cols, int) *matrix {
return &matrix{rows, cols, make([]float, rows*cols)}
}


http://stackoverflow.com/questions/6420734/in-go-how-do-i-create-a-constructor-for-a-type-with-a-string-base-type

type Client struct {
// ...
}

func NewClient() *Client {
return &Client{/* ... */}
}


..."""



//=== new(Type) vs &Type{...} , new(T) and &T{}
https://groups.google.com/forum/#!topic/golang-nuts/9qHAbDDhuqY
https://groups.google.com/forum/#!topic/golang-nuts/FPrkMS2EgoY
http://stackoverflow.com/questions/13244947/is-there-a-difference-between-new-and-regular-allocation

to allocate memory space for an object of 'composite' type T with zero-initial values
--> new T() or &T{}

&T{} doesn't work for basic types,
it is only permitted when T is a struct, array, slice, or map type.
can write "p := new(int)" but can't write "p := &int{0}"

If you want to allocate a pointer,
new(*T) (which returns a **T) is the only way







golang tour basics digest


//===
xxx assignment operator :=
--> := is shortening for declaration and initialization
--> assignment operator is =


//=== http://stackoverflow.com/questions/16521472/assignment-operator-in-go-language
wondering why use
name := "John"

instead of
name = "John"


-->
:= is not the assignment operator,
it serves both as a declaration and as initialization,
"just syntactic sugar"


foo := "bar"
is equivalent to

var foo = "bar"

*** Go is statically typed, so you have to declare variables.


//===
https://tour.golang.org/basics/1
---
https://tour.golang.org/basics/17


* the package name is the same as the last element of the import path.

[ex]
import "math/rand"

the files under "math/rand" folder begin with the statement
package rand


* In Go, a name is exported if it begins with a capital letter.

Foo is an exported name, as is FOO. The name foo is not exported.
[ex]
import "math"
math.Pi is exported and can be used outside "math" package


* function add takes two parameters of type int and return int

func add(x int, y int) int {
return x + y
}




* Inside a function, the := can be used in place of a var declaration with implicit type.
but
Outside a function, ... the := is not available.

[ex]
package main
import "fmt"

x,y := 3.3, 4.4 // --> error
func main() {
var i, j int = 1, 2
k := 3
c, python, java := true, false, "no!"

fmt.Println(i, j, k, c, python, java)
}



* shorten "x int, y int" to "x, y int" in a paramter list or a statement

func add(x, y int) int {
return x + y
}

or

var x,y int = 1,2


or
x,y := 1,2


[ex]
var c, python, java = true, false, "no!" //type inference from values

xxx var t int, s string = 1, "i am a pig"
-->
var t , s = 1, "i am a pig"
t,s:= 1, "i am a pig"

var t int ; s := "i am a pig"



* anonymous return/implicit return/naked return
A return statement without arguments returns the current values of the results.

func split(sum int) (x, y int) {
x = sum * 4 / 9
y = sum - x
return
}






//=== Go's basic types are

bool, string

int int8 int16 int32 int64
uint uint8 uint16 uint32 uint64 uintptr

byte // alias for uint8

rune // alias for int32
// represents a Unicode code point

float32 float64

complex64 complex128




//=== Type conversion
var i int = 42
var f float64 = float64(i)
var u uint = uint(f)

* In Go assignment between items of different type requires an explicit conversion


//=== %T , type inference
fmt.Printf("v is of type %T\n", v)


//===
Constants can be character, string, bool, or numeric values.

Constants cannot be declared using the := syntax.

An untyped constant takes the type needed by its context.

const (
Big = 1 << 100 Small = Big >> 99
)


func main()
{
t:=1
fmt.Println(needFloat(t)) // --> error , need Explicit type conversion
fmt.Println(needFloat(Small))
fmt.Println(needFloat(Big))
}




//=== http://blog.golang.org/gos-declaration-syntax

//=== C
c pointer function
int (*fp)(int a, int b);

Here, fp is a pointer to a function which receives two int parameters and return int.

if one of fp's arguments is itself a function -->
int (*fp)(int (*ff)(int x, int y), int b)


* we can leave out the name of the parameters when we declare a function, so main can be declared

int main(int argc, char *argv[])
--> int main(int, char *[])

drop the name from the middle of its declaration to construct its type.


* what happens to fp's declaration if parameters is unnamed:
int (*fp)(int (*)(int, int), int)
int (*(*fp)(int (*)(int, int), int))(int, int)



* Because C use the same type and declaration syntax, it can be difficult
to parse expressions with types in the middle.
--> C casts always parenthesize the type, as in (int)M_PI


//=== Go

* ?the real main function in Go takes no arguments?


* Here's a declaration of a function variable (analogous to a function pointer in C):

f func(func(int,int) int, int) int
Or if f returns a function:

f func(func(int,int) int, int) func(int, int) int


* Go's type syntax puts the brackets on the left of the type but the expression syntax puts them on the right of the expression:

var a []int
x = a[1]


* Go's pointer syntax is still like the familiar C form, i.e.
var p *int
x = *p

not
x= p*



* Overall, we believe Go's type syntax is easier to understand than C's, especially when things get complicated.


* Go's declarations read left to right.
It's been pointed out that C's read in a spiral! See The "Clockwise/Spiral Rule" by David Anderson.
http://c-faq.com/decl/spiral.anderson.html





docker-machine example for ec2


//=== https://docs.docker.com/machine/

~$ ./docker-machine create ...
~$ ./docker-machine ls
~$ ./docker-machine ip
~$ ./docker-machine rm -f docker-node-name

[port 2376]
* ec2 security group 'docker-machine' needs modification to enhance security
TCP:2376 0.0.0.0/0 --> docker-machine-ip/32
SSH(TCP:22) 0.0.0.0/0 --> docker-machine-ip/32


//=== assume docker-machine is placed under ~/
given ec2 access-key and secret-key

~$ ./docker-machine create --driver "amazonec2"
--amazonec2-vpc-id "vpc-xxxxxxx"
--amazonec2-access-key "xxxxxxxxxxxxxxx"
--amazonec2-secret-key "yyyyyyyyyyyyyyyyyyy"
--amazonec2-region "us-west-1" ec2dk1

[create and launch ec2 instance named 'ec2dk1' (docker-node-name)
with docker daemon bound to port 2376 and
'docker-machine' security group assigned]

$ ./docker-machine ls

$ docker $(./docker-machine config ec2dk1) ps
[err]
2015/03/25 17:34:08 Get http://53.11.233.xxx:2376/v1.12/containers/json: malformed HTTP response "\x15\x03\x01\x00\x02\x02"

??? failed because the docker version installed by
'sudo apt-get install docker.io' is 1.0.1,
which is different from that created by docker-machine on the remote ec2 cloud

-->
~$ wget https://get.docker.com/builds/Linux/x86_64/docker-latest -O docker
~$ chmod +x docker
~$ ./docker $(./docker-machine config ec2dk1) ps


*** to make operation seamless
~$ $(./docker-machine env ec2dk1)
[ now docker points to ec2dk1,
$ echo $DOCKER_TLS_VERIFY
1

$ echo $DOCKER_HOST
tcp://53.11.233.xxx:2376

$ echo $DOCKER_CERT_PATH
/home/u13/.docker/machine/machines/ec2dk1
]

~$ ./docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

~$ ./docker images
~$ ./docker run -it --rm ubuntu:latest bash


//===
~$ ./docker ps
2015/03/25 19:10:58 Cannot connect to the Docker daemon. Is 'docker -d' running on this host?
--> remote docker daemon should be run with -H tcp://node-ip:2376

* docker-machine config.json
$ ls ~/.docker/machine/machines/ec2dk1/*.pem
$ cat ~/.docker/machine/machines/ec2dk1/config.json

{"DriverName":"none","Driver":{"URL":"tcp://54.69.237.148:2376"},
"CaCertPath":"/home/u13/.docker/machine/certs/ca.pem",
"ServerCertPath":"",
"ServerKeyPath":"",
"PrivateKeyPath":"/home/u13/.docker/machine/certs/ca-key.pem",
"ClientCertPath":"",
"SwarmMaster":false,
"SwarmHost":"tcp://0.0.0.0:3376",
"SwarmDiscovery":""}


*** Adding a host without a driver for an existing host
You can add a host to Docker which only has a URL without driver assigned
and can be used an alias for an "existing host" so you don’t have to type out the URL every time.

$ ./docker-machine create --url=tcp://docker-node-ip:2376 ec2dk0
$ docker-machine ls
NAME ACTIVE DRIVER STATE URL
ec2dk0 * none Running tcp://node-ip:2376

--> need ca.pem, ca-key.pem, ...

[Q] how to solve the problem caused by change of ndoe-ip after restarting ec2 instance ?





docker swarm token discovery service


$ swarm create
{token_id} --> unique id to identify the swarm cluster

$ swarm join token://{token_id} --addr=node_ip:port
--> add a docker node to the swarm cluster by its ip:port

[Q] where is the token service for storing the cluster id and member nodes' addresses ?

--> for the time being,

https://discovery-stage.hub.docker.com




//===
~/gocode/src/github.com/docker/swarm/discovery/token$ cat README.md

"""...
#discovery.hub.docker.com

Docker Swarm comes with a simple discovery service built into the [Docker Hub](http://hub.docker.com)

The discovery service is still in alpha stage and currently hosted at `https://discovery-stage.hub.docker.com`

#####Create a new cluster
`-> POST https://discovery.hub.docker.com/v1/clusters (...)`


#####Add new nodes to a cluster
`-> POST https://discovery.hub.docker.com/v1/clusters/token (...)`


#####List nodes in a cluster
`-> GET https://discovery.hub.docker.com/v1/clusters/token`



#####Delete a cluster (all the nodes in a cluster)
`-> DELETE https://discovery.hub.docker.com/v1/clusters/token`




..."""




docker -H


$ sudo docker -H tcp://0.0.0.0:2375 -d
$ sudo docker -H tcp://localhost:2375 -d
$ sudo docker -H tcp://ec2_priv_ip:2375 -d


$ docker --help
"""...
-H, --host=[]
The socket(s) to bind to in daemon mode or connect to in client mode,
specified using one or more tcp://host:port, unix:///path/to/socket, fd://* or fd://socketfd.


..."""



//=== https://docs.docker.com/articles/basics/#bind-docker-to-another-hostport-or-a-unix-socket
"""...
-H accepts host and port assignment in the following format:
tcp://[host][:port]` or `unix://path

...
Run Docker in daemon mode:

$ sudo /docker -H 0.0.0.0:5555 -d &
Download an ubuntu image:

$ sudo docker -H :5555 pull ubuntu


... You can use multiple -H, for example, if you want to listen on both TCP and a Unix socket

# Run docker in daemon mode
$ sudo /docker -H tcp://127.0.0.1:2375 -H unix:///var/run/docker.sock -d &

..."""




docker swarm, node, client or host ?


https://blog.docker.com/2015/02/scaling-docker-with-swarm/
http://devopscube.com/docker-tutorial-getting-started-with-docker-swarm/
http://www.blackfinsecurity.com/getting-started-with-docker-swarm/
http://www.blackfinsecurity.com/docker-swarm-with-tls-authentication/


node === swarm node === docker node within the swarm cluster
swarm client
swarm host
swarm manager

docker client

[Q] how to rm a swarm cluster?
[Q] how to rm a node from swarm?
[Q] On which machine to run swarm join ?




//=== after install swarm by "$ go get -u github.com/docker/swarm"
the 1st terminal
$ swarm create
$ swarm join token:// --addr=54.69.237.148:2375
$ swarm list token://

the 2nd terminal
$ swarm -debug manage --host=swarm-host-ip:swarm-port token://e508ff1181be1e69670d45dd7807a1a2

the 3rd terminal
$ docker -H tcp://swarm-host-ip:swarm-port ps
$ docker -H tcp://swarm-host-ip:swarm-port images
$ docker -H tcp://swarm-host-ip:swarm-port info
$ docker -H tcp://swarm-host-ip:swarm-port run -it --rm ubuntu:latest bash

-->
*** Error response from daemon: No healthy node available in the cluster
*** watch the 2nd terminal(swarm manage) for communication logs between swarm manager and docker nodes


//=== use swarm image from dockerhub, ubuntu14
$ sudo apt-get install docker.io
$ docker pull swarm
$ docker run --rm swarm create
f1c5f2641e8f0346d4dab863415568c4

$

//===
ERRO[0217] Get http://node-ip:2375/v1.15/info: dial tcp node-ip:2375: i/o timeout. Are you trying to connect to a TLS-enabled daemon without TLS?

ERRO[0242] Get http://node-ip:2375/v1.15/info: dial tcp node-ip:2375: i/o timeout. Are you trying to connect to a TLS-enabled daemon without TLS?


-->
https://docs.docker.com/swarm/

TLS
Swarm supports TLS authentication between the CLI and Swarm but also between Swarm and the Docker nodes. However, all the Docker daemon certificates and client certificates must be signed using the same CA-certificate.

In order to enable TLS for both client and server, the same command line options as Docker can be specified:

swarm manage --tlsverify --tlscacert= --tlscert= --tlskey= [...]

Please refer to the Docker documentation for more information on how to set up TLS authentication on Docker and generating the certificates.

Note: Swarm certificates must be generated withextendedKeyUsage = clientAuth,serverAuth.




GOPATH or GOROOT


[Q] GOPATH set to GOROOT has no effect?

//===
http://golang.org/cmd/go/#hdr-GOPATH_environment_variable

"""...
The Go binary distributions assume they will be installed in /usr/local/go (or c:\Go under Windows),
but it is possible to install them in a different location.
If you do this, you will need to set the GOROOT environment variable
to that directory when using the Go tools.

For example, if you installed Go to your home directory you should
add the following commands to $HOME/.profile:

export GOROOT=$HOME/go
export PATH=$PATH:$GOROOT/bin



..."""



//===
http://stackoverflow.com/questions/7970390/what-should-be-the-values-of-gopath-and-goroot
http://golang.org/cmd/goinstall/
http://golang.org/doc/install.html

"""...
GOPATH may be set to a colon-separated list of paths inside which Go code, package objects, and executables may be found.

Set a GOPATH to use goinstall to build and install your own code and external libraries outside of the Go tree (and to avoid writing Makefiles).


... GOROOT The root of the Go tree, often $HOME/go. This defaults to the parent of the directory where all.bash is run. If you choose not to set $GOROOT, you must run gomake instead of make or gmake when developing Go programs using the conventional makefiles.

..."""



install docker on ubuntu

//=== https://docs.docker.com/installation/ubuntulinux/


*** install docker package maintained by docker.com


* Ubuntu Trusty 14.04 (LTS) (64-bit)

0. make sure apt-transport-https is installed

$ [ -e /usr/lib/apt/methods/https ] || {
apt-get update
apt-get install apt-transport-https
}


1. download public key for docker.com repository and append to local key chain(key ring?)
$ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 36A1D7869245C8950F966E92D8576A8BA88D21E9

[ http://manpages.ubuntu.com/manpages/trusty/man8/apt-key.8.html
apt-key :
to manage the list of keys used by apt to authenticate packages.

apt-key adv : pass advanced options to gpg.
apt-key adv --recv-key keyid :
to download the public key corresponding to keyid and add to local keyring

$ man gpg
--recv-keys key IDs
Import the keys with the given key IDs from a keyserver.
Option --keyserver must be used to give the name of this keyserver.

]


2. add the docker.com repo to your apt sources list, update and install the lxc-docker
$ sudo sh -c "echo deb https://get.docker.com/ubuntu docker main > /etc/apt/sources.list.d/docker.list"
$ sudo apt-get update
$ sudo apt-get install lxc-docker


3. verify docker installation
$ docker -v
$ ps aux|grep docker
$ sudo docker run -it ubuntu /bin/bash




//=== https://docs.docker.com/installation/ubuntulinux/

* Ubuntu Precise 12.04 (LTS) (64-bit)

0. upgrade Linux kernel to 3.13+
# install the backported kernel
$ sudo apt-get update
$ sudo apt-get install linux-image-generic-lts-trusty linux-headers-generic-lts-trusty

# install the backported kernel and xorg if using Unity/Xorg
$ sudo apt-get install --install-recommends linux-generic-lts-trusty xserver-xorg-lts-trusty libgl1-mesa-glx-lts-trusty

# reboot
$ sudo reboot

Then
repeat step 1 -- 3 for Ubuntu Trusty 14.04 (LTS) (64-bit)



* Ubuntu Raring 13.04 and Saucy 13.10 (64 bit)

To make sure AUFS is installed, run the following commands:

$ sudo apt-get update
$ sudo apt-get install linux-image-extra-`uname -r`


Then
repeat step 1 -- 3 for Ubuntu Trusty 14.04 (LTS) (64-bit)


* Upgrade docker.com-maintained docker package -- lxc-docker

$ sudo apt-get update
$ sudo apt-get install lxc-docker




*** install docker.io package maintained by ubuntu.com

* Ubuntu Trusty 14.04 (LTS) (64-bit)
"""...
Ubuntu Trusty comes with a 3.13.0 Linux kernel, and a 'docker.io' package which installs 'Docker 1.0.1'

...
Ubuntu (and Debian) contain a much older KDE3/GNOME2 package called docker, so the Ubuntu-maintained package and executable are named docker.io.


... $ sudo apt-get install docker.io [will get 'Docker 1.0.1' ]


..."""

docker, how to rename image


//=== http://stackoverflow.com/questions/25211198/docker-how-to-change-repository-name-or-rename-image
"""...
docker tag server:latest myname/server:latest

or

docker tag d583c3ac45fd myname/server:latest

Tags are just human-readable aliases for the full image name (d583c3ac45fd...).
So you can have as many of them associated with the same image as you like.
If you don't like the old name you can remove it after you've retagged it ...


..."""


$ docker tag old-imgname new-imgname
or
$ docker tag img-id new-imgname

# rm the old imgname when not needed
$ docker rmi old-imgname




Dockerfile, COPY or ADD ?


//=== https://docs.docker.com/articles/dockerfile_best-practices
# Dockerfile
COPY local_files container_folder


"""...
Although ADD and COPY are functionally similar, generally speaking,
COPY is preferred. ...
COPY only supports the basic copying of local files into the container,

...
Because image size matters, using ADD to fetch packages from remote URLs is strongly discouraged;
you should use curl or wget instead. That way you can delete the files you no longer need ...

..."""




//=== http://stackoverflow.com/questions/24958140/docker-copy-vs-add

ADD === COPY + ( tar and remote url handling )






docker install by binary


//=== https://docs.docker.com/installation/binaries/

"""...
Get the docker binary:
$ wget https://get.docker.com/builds/Linux/x86_64/docker-latest -O docker
$ chmod +x docker

... you can also get the smaller compressed release file:
https://get.docker.com/builds/Linux/x86_64/docker-latest.tgz

Run the docker daemon
...
$ sudo ./docker -d &

....
Upgrades
To upgrade your manual installation of Docker, first kill the docker daemon:

$ killall docker

..."""





Dockerfile RUN


//=== https://docs.docker.com/reference/builder/
"""...
RUN has 2 forms:

RUN (the command is run in a shell - /bin/sh -c - shell form)
RUN ["executable", "param1", "param2"] (exec form)

... To use a different shell, other than '/bin/sh', use the exec form passing in the desired shell.
For example, RUN ["/bin/bash", "-c", "echo hello"]

... The exec form is parsed as a JSON array, which means that you must use double-quotes (") around words
not single-quotes (').

... Unlike the shell form, the exec form does not invoke a command shell.
This means that normal shell processing does not happen.
For example, RUN [ "echo", "$HOME" ] will not do variable substitution on $HOME.



..."""


RUN cp src/file1 dest/file1




How to tell git which private key to use for ssh-git docker container


//=== http://superuser.com/questions/232373/how-to-tell-git-which-private-key-to-use
[Q] how to tell git which private key to use?

* edit ~/.ssh/config to add the host-key mapping for git server
[
Hostname git-server-ip-or-hostname
IdentityFile path-to-private-key
]

* Write a script that calls ssh with -i path-to-priv-key, and place script filename in $GIT_SSH. ???

* use ssh-agent to temporary authorize a private key.

??? ssh-agent sh -c 'ssh-add path-to-priv-key; git fetch user@git-sever'

$ ssh-add path-to-priv-key
[err] Could not open a connection to your authentication agent.


//=== http://stackoverflow.com/questions/17846529/could-not-open-a-connection-to-your-authentication-agent/20403535

-->
using CentOS.
$ exec ssh-agent bash

--> git bash
$ eval $(ssh-agent)

$ ssh-add path-to-priv-key
$ ssh -v git@container-ip
git> create mygitssh


//=== after create a repository mygitssh.git on git-server
[
$ git clone git@container-ip:mygitssh.git
...
$ git add .
$ git push origin master
]

or

[
$ git init
...
$ git add .

$ git remote add origin git@container-ip:mygitssh.git
$ git commit -m "init"
]




docker data volume


//=== https://docs.docker.com/userguide/dockervolumes/#volume

"""...
A data volume is a specially-designated directory within one or more containers
that bypasses the Union File System.

...
Data volumes are designed to persist data, independent of the container's life cycle.
Docker therefore never automatically delete volumes when you remove a container,
nor will it "garbage collect" volumes that are no longer referenced by a container.


...
You can add a data volume to a container using the -v flag with the docker create and docker run command.
You can use the -v multiple times to mount multiple data volumes

...
You can also use the VOLUME instruction in a Dockerfile to add one or more new volumes to any container
created from that image.


...
The -v flag can also be used to mount a single file - instead of just directories - from the host machine.

$ sudo docker run --rm -it -v ~/.bash_history:/.bash_history ubuntu /bin/bash


...
Creating and mounting a Data Volume Container

...
Let's create a new named container with a volume to share.
While this container doesn't run an application,
it reuses the training/postgres image so that ... saving disk space.

$ sudo docker create -v /dbdata --name dbdata training/postgres

...
$ sudo docker run -d --volumes-from dbdata --name db1 training/postgres

$ sudo docker run -d --volumes-from dbdata --name db2 training/postgres

... The result is only the files from the dbdata container are visible.

... You can use multiple --volumes-from parameters to bring together multiple data volumes from multiple containers.


$ sudo docker run -d --name db3 --volumes-from db1 training/postgres

... To delete the volume from disk, you must explicitly call
docker rm -v against the last container with a reference to the volume.



..."""




docker image for git+ssh


https://registry.hub.docker.com/u/phylor/git/dockerfile/
https://github.com/phylor/git-docker

docker repo for git+ssh

can use ubuntu:14.04 as base image:

# === Dockerfile
FROM ubuntu:14.04
RUN DEBIAN_FRONTEND=noninteractive apt-get update
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y ssh git
[install git and ssh]
[ssh === openssh-server + openssh-client ]
...
[ adduser git with home folder /home/git for restricted git shell]
[ create folder /var/run/sshd, set proper permission ]
[ create folder /home/git/.ssh, set proper permission ]
[ create file /home/git/.ssh/authorized_keys, set proper permission]
[ create folder /home/git/repositories]
[ create folder /home/git/git-shell-commands ]

[ copy list, create, start from https://github.com/phylor/git-docker to local,
set proper permission]

# RUN mkdir /keys ?
# VOLUME /keys ?

CMD /start

# ================


$ docker build -t gitssh:t1 .
[ create image from Dockerfile ]

$ docker run -it --rm -v pubkey:/keys -p 3322:22 --name mygitssh gitssh:t1

$ docker inpsect mygitssh|grep IPAddress
$ ssh -v -i -p 22 pubkey git@container_ip






ec2, access key and secret key


http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSGettingStartedGuide/AWSCredentials.html

[for short]
access key ID --> access key
secret access key --> secret key


"""...
To get your access key ID and secret access key
... We recommend that you use IAM access keys instead of AWS root account access keys.
IAM lets you securely control access to AWS services and resources in your AWS account.

... Open the IAM console.
From the navigation menu, click Users.

Select your IAM user name.

Click User Actions, and then click Manage Access Keys.

Click Create Access Key.

Your keys will look something like this:

Access key ID example: AKIAIOSFODNN7EXAMPLE

Secret access key example: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

Click Download Credentials, and store the keys in a secure location.

*** Your secret key will no longer be available through the AWS Management Console;
... the only copy.
... never email it.
Do not share it outside your organization,
even if an inquiry appears to come from AWS or Amazon.com.
...

..."""


[ like the private key of ssh key-pair , the secret key only appeared once when created
and will not be stored on amazon web site ]




ssh troubles



[Q] why passwordless ssh login still ask for user password?


-->
* ssh -v
* chk local, .ssh folder permission is 700
* cat /var/log/auth.log
* cat /var/log/security
[??? cat: /var/log/security: No such file or directory ]

* chk remote, ~/.ssh/authorized_keys file permission is 400 or 600,
* chk local, private key file is 600

* chk spelling of filename, 'authorized_keys' file, and /etc/ssh/sshd_config

* chk the user login shell by
$ cat /etc/passwd |grep username


//=== change the login shell for a certain user

* change the /etc/passwd file directly
or
$ chsh -s /bin/bash username

Then log in and log out


//=== /etc/ssh/sshd_config
* PermitRootLogin

The argument must be “yes”, “without-password”, “forced-commands-only”, or "no”.
*** The default is “yes”.

“without-password” will disable password authentication for root.
but allow root login only with public key authentication.


* AllowUsers root user1 user2 user3

Any other users than the ones listed above trying to login via SSH would receive the error messages:
"Roaming not allowed by server"


* PermitEmptyPasswords no
http://stackoverflow.com/a/14421105/758174
??? make sure the account you are trying to ssh to is a user with a password on the remote server

* UsePAM
ssh failed to login because PAM rules check failure
* chk the /etc/pam.d/sshd for rules applied
* common problem is a user without password
[ compare the /etc/passwd with /etc/shadow]
[ chk your /etc/nsswitch and /etc/pam.d/* to see auth source]
* missing home directory,
* missing some extra auth configuration
* UID too low or too high



//=== ssh -v
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: password
git@172.187.0.40's password:
debug1: Authentications that can continue: publickey,password

...




//=== https://www.cs.utah.edu/~bigler/code/sshkeys.html

"... One of the login modes of ssh is to use a SSH key pair.
A key pair is made up of both a private and a public key.
The private key is kept on your local machine while your
public key is what you distribute to ...

There are a few flavors of keys you can generate,
rsa1 (for SSH1), dsa (SSH2), or rsa (SSH2).

According to my IT guy he likes DSA.
You can (and should) associate a password with your key pair ..."""
[not the user account password, just a challenge password for key pair]

"" ...
You can also vary the number of bits used for the key...
I was recommended to use 2048 bits.
..."""

$ ssh-keygen -t dsa -b 2048




//=== ssh2.pem is just an example for private key filename
xxxx $ scp -i ssh2.pem ./* xyz.com:workspaces/data/
-->
$ scp -i ssh2.pem * username@xyz.com:workspaces/data/


xxxx $ ssh -i privkey xyz.com
-->
$ ssh -i privkey * username@xyz.com

[if username@ is omitted, the current user under local machine shell will used ...]



docker cgroup

//=== https://blog.docker.com/2013/10/gathering-lxc-docker-containers-metrics/

*** Control groups are exposed via a pseudo-filesystem 'cgroup'


cgroup location:

/sys/fs/cgroup
or
/cgroup

or find it by
$ grep cgroup /proc/mounts


"""...
different control groups can be in different hierarchies
mean that you can use completely different groups (and policies)
for e.g. CPU allocation and memory allocation.
..."""

$ cat /proc/cgroups





*** to find the memory configs via cgroup for a given container
--> /sys/fs/cgroup/memory/lxc//.

-->
$ cd /sys/fs/cgroup/memory/docker/f177fc8c6f0c208f50...
$ cat memory.stat
...
hierarchical_memory_limit 92233720368
...
$ cat memory.limit_in_bytes
$ cat memory.swappiness



???
"""...
the memory and memsw limits are not really metrics,
but a reminder of the limits applied to this cgroup.
The first one indicates the maximum amount of physical memory that can be used by the processes of this control group;
the second one indicates the maximum amount of RAM+swap.
..."""



//=== short_id to long_id
a container shows short_id as ae836c95b4c3 by
$ docker ps -a

its long_id might be something like ae836c95b4c3c9e9179e0e91015...
-->
$ docker inspect
or
$ docker ps -notrunc




//===
docker run -m
docker run --memory

$ docker run -it --rm --memory 2000M python:2.7



//=== http://stackoverflow.com/questions/20096632/limit-memory-on-a-docker-container-doesnt-work
"""...
$ docker run -m=524288 -d -t busybox sleep 3600

...
$ cat /sys/fs/cgroup/memory/lxc/f03a017b174ff1022e0f46bc1b307658c2d96ffef1dd97e7c1929a4ca61ab80f//memory.limit_in_bytes

... Docker's -m flag does not count swap and ram separately. If you use -m 512M then some of that 512 will be swap, not RAM.

# Same as docker -m 512m
sudo docker run --lxc-conf="lxc.cgroup.memory.limit_in_bytes=512M" -it ubuntu /bin/bash

# Set total to equal max RAM (e.g. don't use swap)
sudo docker run --lxc-conf="lxc.cgroup.memory.max_usage_in_bytes=512M" --lxc-conf="lxc.cgroup.memory.limit_in_bytes=512M" -it ubuntu /bin/bash

..."""

docker export/import, pull/commit/push, save/load


//=== http://tuhrig.de/difference-between-save-and-export-in-docker/
pull image busybox
start a container from busybox image
add a new folder /home/test on the running container
commit the running container to another image busybox-1

image "busybox-1" contains the changes done to "busybox"

""" ...
... can create a new image from a running container (and all it changes)
using docker commit container-id image-name

...
the exported-imported image has lost all of its history
whereas
the saved-loaded image still have its history and layers.

...

Do I need to stop container defore export it to tar file?
... No, you should also be able to export a running container.

..."""



//=== export/import targets container
$ docker export container1 > c1.tar
$ cat c1.tar | docker import - c1-export:latest
[the result of import is a newly created image "c1-export:latest"]
[xxx need repository server xxx]


//=== save/load targets image
$ docker save img1 > i1.tar
$ docker load < i1.tar //=== ??? $ docker images --tree imgname [Warning: '--tree' is deprecated, it will be removed soon. See usage.] --> $ docker images -a imgname


//===
$ docker export -h
$ docker save -h
$ docker commit -h [xxx need repository server xxx]


//=== "docker push" need repository server

$ docker push ipynb:t4
--> You cannot push a "root" repository.
Please rename your repository in username/reponame (ex: u1333/ipynb:t4)


to rename image
docker tag ipynb:t4 username/reponame:t4

or

docker tag img_id username/reponame:t4





git push -u or not

$ git push origin master
$ git push -u origin master

-->
http://ubuntu.elfscript.com/2015/03/git-push-u.html

docker commandline examples

//===
$ docker pull repo:tag
$ docker commit -m="msg" container_name_or_id dockerhub_username/repo:tag

$ docker commit --help


# show process for container
$ docker ps
$ docker ps -a
$ docker rm container-name
$ docker rm container-id


# docker top --help
Usage: docker top CONTAINER [ps OPTIONS]
Lookup the running processes of a container




# build image from the current directory
$ docker build -t imgname .

$ docker images
$ docker rmi img-tag
$ docker rmi img-id

# run an image , start/stop container
$ docker run -p 27017:27017 -d --name container_name imgname
$ docker stop container_name
$ docker start container_name

$ docker run -it --rm --name mypy python:2.7 bash
http://it.3hd.me/2015/03/docker-image-for-python.html

//=== daemon
# in Ubuntu 14.04, use docker.io instead of docker
$ sudo service docker.io restart
$ sudo docker.io -d & [???]

docker without sudo


//=== https://docs.docker.com/installation/ubuntulinux/#giving-non-root-access
The docker daemon must always run as the root user,
but
if you run the docker client as a user in the "docker" group
then you don't need to add sudo to all the client commands.


The docker daemon runs as the root user and
binds to a Unix socket instead of a TCP port.
[ By default that Unix socket is owned by the user root
and you can access it with sudo.]

*** create a group "docker" and add your desired user to it,
[ the docker daemon will make the ownership of the Unix socket
read/writable by the "docker" group when the daemon starts. ]


From Docker 0.9.0, use the -G flag to specify an alternative group to "docker".
[
$ man docker.io
$ docker.io --help
$ sudo docker.io -G grpdocker -d
]

*** The docker group (or the group specified with the -G flag) is root-equivalent
--> Docker Daemon Attack

//===
# Add the docker group if it doesn't already exist.
$ sudo groupadd docker

# Add the current user "${USER}" to the docker group.
$ sudo gpasswd -a ${USER} docker


$ exit
# then login again


# Restart the Docker daemon.
$ sudo service docker restart

[Q]how to restart dokcer daemon on ubuntu???

*** docker daemon name is different between Ubuntu 12 and Ubuntu 14
# If you are in Ubuntu 14.04, use docker.io instead of docker
$ sudo service docker.io restart


then
$ docker run -it --rm --name container_name img_name cmd
$ docker ps
$ docker images
$ docker build -t img_name ./folder_to_be_included
...



貨到付款, 代收貨款


貨到付款
到付宅急便(只有代收運費) 與 貨到付款 不同 !

黑貓尚未整合貨到付款
7/11的交貨便限制了長寬高的和要小於90cm, 最長邊要小於45cm ?
--> 郵局


//=== http://www.collect-service.com.tw/service.php?type=expense&id=106
4.黑貓宅急便有提供貨到付款或貨到刷卡服務嗎？
黑貓宅急便目前只提供運費由收件人付款的服務(即到付宅急便服務)不包括商品費用的付款服務，有關商品費用的付款問題請洽詢宅急便客樂得公司


//=== http://www.post.gov.tw/post/internet/Postal/index.jsp?ID=1397437571086
郵局「代收貨款」服務結合郵局安全便捷的物流、金流系統，提供您從
「窗口收寄 / 上門收件(※) / 物流倉儲中心出貨」→「遞送」→「貨款收取」→「貨款24hr內入帳」之完整「貨到付款」服務。


目前有2種代收貨款入帳方式可供選擇：
A.撥存於寄件人之劃撥儲金帳戶 →除應加收之資費外，免收匯費。

不逾5,000元 無論國內函件、包裹、快捷郵件，除普通及其他特種資費外，每件另加 30元

B. ...



//=== 郵局國內包裹 尺寸限制：
最小尺寸:不得小於14x9公分。成捲者:其長度及直徑之2倍，合計不得小於17公分，其最大一面之尺寸不得小於10公分。
最大尺寸:單邊不超過150公分，每件長度加長度以外最大周圍合計亦不得超過3公尺(即:長x1+寬x2+高x2不逾300公分)。

[運費]
http://www.post.gov.tw/post/internet/Postal/index.jsp?ID=2030103
本島同縣市 北北基 本島不同縣市
不逾5公斤 70 80 100

逾5公斤
不逾10公斤 90 100 125



mongodb user/password


http://docs.mongodb.org/manual/tutorial/enable-authentication/


[authentication]
to start a mongod with authentication enabled and
a key file stored in /path-to-keyfile ,
set the following option in the mongod‘s config file:

security:
keyFile: /path-to-keyfile

Then start the mongod with config file specified :

mongod --config /etc/mongodb/mongodb.conf



[authorization]
??? security.authorization
Type: string
Default: disabled
Enables Role-Based Access Control (RBAC) to govern each user’s access to database resources and operations.





//=== Err msg: """ Property 'createUser' of object admin is not a function """

--> for MongoDB 2.4, use db.addUser()

db.createUser() is for 2.6+
since 2.6, MongoDB introduced a new model for user credentials and privileges;




//===
http://docs.mongodb.org/manual/core/authentication/#localhost-exception


To disable the localhost exception,
??? use setParameter to set the enableLocalhostAuthBypass parameter to 0 during startup.



//=== http://docs.mongodb.org/manual/tutorial/add-user-administrator/
""" ...
to restart mongod with the --noauth option to create a new user with system admin privilege
..."""



//=== Create the system administrator and database admin
* use admin database, then create user

> use admin
> db.createUser(
{
user: "sysAdmin",
pwd: "syspasswd",
roles: [ { role: "userAdminAnyDatabase", db: "admin" } ]
}
)



* Create a user admin for a single database.
> use db1
> db.createUser(
{
user: "db1Admin",
pwd: "db1passwd",
roles: [ { role: "userAdmin", db: "db1" } ]
}
)

>use db2
>db.createUser(
{
user: "db2Admin",
pwd: "db2passwd",
roles: [ { role: "userAdmin", db: "db2" } , {role:"read", db:"db1"}]
}
)


//=== http://docs.mongodb.org/manual/tutorial/add-user-to-database/

"""...
use reporting
db.createUser(
{
user: "reportsUser",
pwd: "12345678",
roles: [
{ role: "read", db: "reporting" },
{ role: "read", db: "products" },
{ role: "read", db: "sales" },
{ role: "readWrite", db: "accounts" }
]
}
)

..."""




docker image for python


https://registry.hub.docker.com/_/python/

$ docker pull python:2.7
or
$ docker pull python:2.7.9 ???


$ docker images
python 2.7 31ff30c97af1 3 weeks ago 745.2 MB


$ docker run -it --rm --name mypy python:2.7
[python console]


$ docker run -it --rm --name mypy python:2.7 bash
[bash shell]

root@83810d5676d0:/# pip --version
pip 6.0.8 from /usr/local/lib/python2.7/site-packages (python 2.7)

root@83810d5676d0:/# virtualenv --version
12.0.7

root@83810d5676d0:/# python --version
Python 2.7.9



//=== https://github.com/docker-library/python/blob/25134c1757699a357936b94924a97536f9526040/2.7/Dockerfile

* python:2.7 Dockerfile based on "jessie"

"""...
FROM buildpack-deps:jessie

# remove several traces of debian python
RUN apt-get purge -y python.*

# http://bugs.python.org/issue19846
# > At the moment, setting "LANG=C" on a Linux system *fundamentally breaks Python 3*, and that's not OK.
ENV LANG C.UTF-8

ENV PYTHON_VERSION 2.7.9

# gpg: key 18ADD4FF: public key "Benjamin Peterson " imported
RUN gpg --keyserver pool.sks-keyservers.net --recv-keys C01E1CAD5EA2C4F0B8E3571504C367C218ADD4FF

...


..."""


//=== https://github.com/docker-library/python/blob/d550e292eec57e83af58e05410243d387d6483a8/2.7/onbuild/Dockerfile

* python:2.7-onbuild Dockerfile based on python:2.7.9

"""
FROM python:2.7.9

RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app

ONBUILD COPY requirements.txt /usr/src/app/
ONBUILD RUN pip install -r requirements.txt

ONBUILD COPY . /usr/src/app

"""


//=== python:2.7.9-wheezy, 2.7-wheezy, 2-wheezy (2.7/wheezy/Dockerfile)
python image based on "wheezy"

FROM buildpack-deps:wheezy





docker resource management, memory limit

//===
https://docs.docker.com/reference/commandline/cli/#run
https://docs.docker.com/reference/run/#runtime-constraints-on-cpu-and-memory

-c === --cpu-shares [cpu weighting]
--cpu [number of cpu cores]
--cpuset [set of cpu cores]

-m [memory limit]

-memory-swap=""
[Total memory usage (memory + swap), set '-1' to disable swap ]

$ docker run -it --rm -m 128m -c 512 image_name

$ docker top container_name
$ docker


//=== https://goldmann.pl/blog/2014/09/11/resource-management-in-docker/#_example_managing_the_cpu_shares_of_a_container

?? --vm , --vm-bytes, --vm-hang

"""...
Every new container will have 1024 shares of CPU by default. This value does not mean anything, when speaking of it alone. But if we start two containers and both will use 100% CPU, the CPU time will be divided equally between the two containers because they both have the same CPU shares (for the sake of simplicity I assume that there are no other processes running).

If we set one container’s CPU shares to 512 it will receive half of the CPU time compared to the other container. But this does not mean that it can use only half of the CPU. If the other container (with 1024 shares) is idle — our container will be allowed to use 100% of the CPU. That’s another thing to note.


...
To allow execution only on the first core:

docker run -it --rm --cpuset=0 stress --cpu 1

To allow execution only on the first two cores:

docker run -it --rm --cpuset=0,1 stress --cpu 2


...
$ docker run -it --rm -m 128m stress --vm 1 --vm-bytes 128M --vm-hang 0
stress: info: [1] dispatching hogs: 0 cpu, 0 io, 1 vm, 0 hdd
The stress tool will create one process and try to allocate 128MB of memory to it.
It works fine, good. But what happens if we try to use more than we have actually allocated for the container?

$ docker run -it --rm -m 128m stress --vm 1 --vm-bytes 200M --vm-hang 0
stress: info: [1] dispatching hogs: 0 cpu, 0 io, 1 vm, 0 hdd
It works too. Surprising? ...

... by default the memory.memsw.limit_in_bytes value is set to twice as much as the memory parameter we specify...
... What does the memory.memsw.limit_in_bytes parameter say? It is a sum of memory and swap.
This means that Docker will assign to the container -m amount of memory as well as -m amount of swap.

...
With the above information we can run our example again.
This time we will try to allocate over twice the amount of memory we assign.
This should use all of the memory and all of the swap, then die.

$ docker run -it --rm -m 128m stress --vm 1 --vm-bytes 260M --vm-hang 0
stress: info: [1] dispatching hogs: 0 cpu, 0 io, 1 vm, 0 hdd

stress: FAIL: [1] (415) <-- worker 6 got signal 9 stress: WARN: [1] (417) now reaping child worker processes ...

..."""

Dockerfile example for nodejs

//=== https://docs.docker.com/examples/nodejs_web_app/
"""...

FROM centos:centos6

# Enable EPEL for Node.js
RUN rpm -Uvh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm
# Install Node.js and npm
RUN yum install -y npm

# Bundle app source
COPY . /src
# Install app dependencies
RUN cd /src; npm install

EXPOSE 8080
CMD ["node", "/src/index.js"]



..."""