%runscript echo "This is what happens when you run the container..." bash %setup # 1）这一步可以先设置好环境变量，通过wget等下载好必要的软件库等，注意这些指令都只是在host机器上运行，不是container里面。 # 内容请参考https://git.oschina.net/sg-ai/singularityimages/blob/master/keras-tf-1.0.1-gpu.setup %post # non interactive debian DEBIAN_FRONTEND=noninteractive # Install the necessary packages (from repo) apt-get update && apt-get install -y --no-install-recommends curl ca-certificates
GLOBAL OPTIONS: -d --debug Print debugging information -h --help Display usage summary -q --quiet Only print errors --version Show application version -v --verbose Increase verbosity +1 -x --sh-debug Print shell wrapper debugging information
GENERAL COMMANDS: help Show additional help for a command
CONTAINER USAGE COMMANDS: exec Execute a command within container run Launch a runscript within container shell Run a Bourne shell within container test Execute any test code defined within container
CONTAINER MANAGEMENT COMMANDS (requires root): bootstrap Bootstrap a new Singularity image from scratch copy Copy files from your host into the container create Create a new container image expand Grow the container image export Export the contents of a container via a tar pipe import Import/add container contents via a tar pipe mount Mount a Singularity container image
Obtain a shell (/bin/sh) within the container image.
note: When invoking a shell within a container, the container image is by default writable.
SHELL OPTIONS: -B/--bind <spec> A user-bind path specification. spec can either be a path or a src:dest pair, specifying the bind mount to perform (can be called multiple times) -c/--contain This option disables the automatic sharing of writable filesystems on your host (e.g. $HOME and /tmp). -C/--containall Contain not only file systems, but also PID and IPC -H/--home <path> Path to a different home directory to virtualize within the container -i/--ipc Run container in a new IPC namespace -p/--pid Run container in a new PID namespace (creates child) --pwd Initial working directory for payload process inside the container -S/--scratch <path> Include a scratch directory within the container that is linked to a temporary dir (use -W to force location) -s/--shell <shell> Path to program to use for interactive shell -u/--user Try to run completely unprivileged (only works on very new kernels/distros) -W/--workdir Working directory to be used for /tmp, /var/tmp and $HOME (if -c/--contain was also used) -w/--writable By default all Singularity containers are available as read only. This option makes the file system accessible as read/write.
NOTE: If there is a daemon process running inside the container, then subsequent container commands will all run within the same namespaces. This means that the --writable and --contain options will not be honored as the namespaces have already been configured by the 'singularity start' command.
$curl http://172.16.10.10:5000/v2/paddlepaddle/paddle/tags/list|python2 -m json.tool % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 84 100 84 0 0 20771 0 --:--:-- --:--:-- --:--:-- 28000 { "name": "paddlepaddle/paddle", "tags": [ "latest", "0.10.0", "0.10.0-dev", "0.10.0-gpu" ] }
$docker pull 172.16.10.10:5000/paddlepaddle/paddle:0.10.0 #下载images ... $docker images REPOSITORY TAG IMAGE ID CREATED SIZE python 2.7 26bddf7dbe1b 6 days ago 679MB $docker tag python:2.7 172.16.10.10:5000/python:2.7 $docker images REPOSITORY TAG IMAGE ID CREATED SIZE 172.16.10.10:5000/python 2.7 26bddf7dbe1b 6 days ago 679MB python 2.7 26bddf7dbe1b 6 days ago 679MB $docker push 172.16.10.10:5000/python:2.7 #上传images The push refers to a repository [172.16.10.10:5000/python] c89798427b75: Pushed ... 18f9b4e2e1bc: Pushed 2.7: digest: sha256:d3ecae8689444b025ff6dfd0972d98e4c0f9e8b9ca5762f044447dfd654c1b53 size: 2011 [superman@node7 ~]$curl http://172.16.10.10:5000/v2/_catalog|python2 -m json.tool { "repositories": [ ... "python", ... ] }
WARNING: Converting logical volume docker/thinpool and docker/thinpoolmeta to thin pool's data and metadata volumes with metadata wiping. THIS WILL DESTROY CONTENT OF LOGICAL VOLUME (filesystem etc.) Converted docker/thinpool to thin pool.
$ sudo vi /etc/lvm/profile/docker-thinpool.profile #配置比例,其实不写也可以 activation { thin_pool_autoextend_threshold=80 thin_pool_autoextend_percent=20 }
Usage: docker image COMMAND Commands: ls List images pull Pull an image or a repository from a registry push Push an image or a repository to a registry rm Remove one or more images save Save one or more images to a tar archive (streamed to STDOUT by default) load Load an image from a tar archive or STDIN tag Create a tag TARGET_IMAGE that refers to SOURCE_IMAGE build Build an image from a Dockerfile Run 'docker image COMMAND --help' for more information on a command.
1.docker images
列出主机上已存在镜像
1 2 3 4 5 6 7
$ docker images REPOSITORY TAG IMAGE ID CREATED SIZE 172.16.10.10:5000/deepchem cpu 75105fd5d68d 11 days ago 6.98GB 172.16.10.10:5000/deepchem 2.0.0-cpu-worker 2b024a901b22 2 weeks ago 6.75GB 172.16.10.10:5000/python-3.5.4-alpine v3 954d2aa346fb 2 months ago 309MB deepchemio/deepchem 2.0.0-cpu d0b0679dd7ce 3 months ago 6.02GB crossbario/crossbar latest 01c84a0d6626 3 months ago 170MB
docker save hooby/deepchem:2.0.0-cpu -o deepchem.tar
6.docker load
导入image
1
docker load -i deepchem.tar
7.docker history
查看image构建过程中的命令历史
1 2 3 4 5
docker history deepchemio/deepchem:2.0.0-cpu IMAGE CREATED CREATED BY SIZE COMMENT d0b0679dd7ce 3 months ago /bin/sh -c cd deepchem && git clean -fX 0B <missing> 3 months ago /bin/sh -c export LANG=en_US.UTF-8 && gi… 4.31GB ...
Usage: docker container COMMAND Commands: ls List containers run Run a command in a new container start Start one or more stopped containers restart Restart one or more containers exec Run a command in a running container rm Remove one or more containers kill Kill one or more running containers stop Stop one or more running containers logs Fetch the logs of a container rename Rename a container commit Create a new image from a container's changes
Run 'docker container COMMAND --help' for more information on a command.
$ docker ps #显示正在运行状态下的container CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 10cf2cd2d507 172.16.10.10:5000/deepchem:cpu "bash" 9 days ago Up 6 days deepchem-cpu 18b35ff44321 crossbario/crossbar "crossbar start --cb…" 9 days ago Up 6 days 8000/tcp, 0.0.0.0:8080->8080/tcp crossbar-test $ docker ps -a #显示所有container CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 10cf2cd2d507 172.16.10.10:5000/deepchem:cpu "bash" 9 days ago Up 6 days deepchem-cpu 18b35ff44321 crossbario/crossbar "crossbar start --cb…" 9 days ago Up 6 days 8000/tcp, 0.0.0.0:8080->8080/tcp crossbar-test 5c0416e9e3d1 crossbario/crossbar:latest "crossbar start --cb…" 9 days ago Exited (1) 9 days ago crossbar_test 8caf97e2c03f 172.16.10.10:5000/deepchem:2.0.0-cpu-worker "bash" 11 days ago Exited (0) 10 days ago deepchem
2.docker run
从image启动，运行一个container
1 2 3
docker run deepchemio/deepchem:2.0.0-cpu docker run -d --name deepchem deepchemio/deepchem:2.0.0-cpu docker run -d --name crossbar --rm -v node_config:/node -p 8080:8080 crossbario/crossbar
一个视图函数（或简称为视图）是一个 Python 函数，它接受 Web 请求并返回一个 Web 响应。这个响应可以是 Web 页面的 HTML 内容，或者重定向，或者404错误，或者 XML 文档，或一个图片…或是任何内容。视图本身包含返回响应所需的任何逻辑。这个代码可以存在任何地方，只要它在你的 Python 路径上就行。可以说，不需要其他东西，这里并没有魔法。为了将代码放置在某处，约定将视图放在名为 views.py 的文件里，这个文件放置在项目或应用目录里。
<a href="{% url 'news-year-archive' 2012 %}">2012 Archive</a> {# Or with the year in a template context variable: #} <ul> {% for yearvar in year_list %} <li><a href="{% url 'news-year-archive' yearvar %}">{{ yearvar }} Archive</a></li> {% endfor %} </ul>
或在 Python 代码里：
1 2 3 4 5 6 7 8
from django.http import HttpResponseRedirect from django.urls import reverse
# Total number of books. >>> Book.objects.count() 2452
# Total number of books with publisher=BaloneyPress >>> Book.objects.filter(publisher__name='BaloneyPress').count() 73
# Average price across all books. >>> from django.db.models import Avg >>> Book.objects.all().aggregate(Avg('price')) {'price__avg': 34.35}
# Max price across all books. >>> from django.db.models import Max >>> Book.objects.all().aggregate(Max('price')) {'price__max': Decimal('81.20')}
# Difference between the highest priced book and the average price of all books. >>> from django.db.models import FloatField >>> Book.objects.aggregate( ... price_diff=Max('price', output_field=FloatField()) - Avg('price')) {'price_diff': 46.85}
# All the following queries involve traversing the Book<->Publisher # foreign key relationship backwards.
# Each publisher, each with a count of books as a "num_books" attribute. >>> from django.db.models import Count >>> pubs = Publisher.objects.annotate(num_books=Count('book')) >>> pubs <QuerySet [<Publisher: BaloneyPress>, <Publisher: SalamiPress>, ...]> >>> pubs[0].num_books 73
# Each publisher, with a separate count of books with a rating above and below 5 >>> from django.db.models import Q >>> above_5 = Count('book', filter=Q(book__rating__gt=5)) >>> below_5 = Count('book', filter=Q(book__rating__lte=5)) >>> pubs = Publisher.objects.annotate(below_5=below_5).annotate(above_5=above_5) >>> pubs[0].above_5 23 >>> pubs[0].below_5 12
# The top 5 publishers, in order by number of books. >>> pubs = Publisher.objects.annotate(num_books=Count('book')).order_by('-num_books')[:5] >>> pubs[0].num_books 1323
# Build an annotated queryset >>> from django.db.models import Count >>> q = Book.objects.annotate(Count('authors')) # Interrogate the first object in the queryset >>> q[0] <Book: The Definitive Guide to Django> >>> q[0].authors__count 2 # Interrogate the second object in the queryset >>> q[1] <Book: Practical Django Projects> >>> q[1].authors__count 1
>>> Blog.objects.get(id__exact=14) # Explicit form >>> Blog.objects.get(id=14) # __exact is implied >>> Blog.objects.get(pk=14) # pk implies id__exact
# Get blogs entries with id 1, 4 and 7 >>> Blog.objects.filter(pk__in=[1,4,7]) # Get all blog entries with id > 14 >>> Blog.objects.filter(pk__gt=14)
pk 查找也支持跨连接。
1 2 3
>>> Entry.objects.filter(blog__id__exact=3) # Explicit form >>> Entry.objects.filter(blog__id=3) # __exact is implied >>> Entry.objects.filter(blog__pk=3) # __pk implies __id__exact
缓存和 QuerySet
每个 QuerySet 都带有缓存，尽量减少数据库访问。
例如，以下会创建两个 QuerySet，计算它们，丢掉它们:
1 2
>>> print([e.headline for e in Entry.objects.all()]) >>> print([e.pub_date for e in Entry.objects.all()])
两个结果：
同样的数据库查询会被执行两次，实际加倍了数据库负载。
有可能这两个列表不包含同样的记录，因为在两次请求间，可能有 Entry 被添加或删除了。
要避免此问题，保存 QuerySet 并复用它:
1 2 3
>>> queryset = Entry.objects.all() >>> print([p.headline for p in queryset]) # Evaluate the query set. >>> print([p.pub_date for p in queryset]) # Re-use the cache from the evaluation.
b = Blog.objects.get(pk=1) # This will delete the Blog and all of its Entry objects. b.delete()
复制模型实例
将 pk 设为 None。
1 2 3 4 5
blog = Blog(name='My blog', tagline='Blogging is easy') blog.save() # blog.pk == 1
blog.pk = None blog.save() # blog.pk == 2
一次修改多个对象
1 2 3 4 5 6 7
# Update all the headlines with pub_date in 2007. Entry.objects.filter(pub_date__year=2007).update(headline='Everything is the same') >>> b = Blog.objects.get(pk=1)
# Change every Entry so that it belongs to this Blog. >>> Entry.objects.all().update(blog=b)
class Person(models.Model): first_name = models.CharField(...) last_name = models.CharField(...) birth_date = models.DateField(...) >>> for p in Person.objects.raw('SELECT * FROM myapp_person'): ... print(p) John Smith Jane Jones