HyperAI configuration file

Provide default parameters for execution

HyperAI configuration file (HyperAI.yaml)Used in conjunction with command-line tools. Can greatly simplify the input of repetitive commands.

When using command-line tools. You can use command parameters to specify the required "environment" "resources" "data set" Wait for the information to be transmitted, for example:

bayes gear run task \
    --env=tensorflow-1.12 \ # Specify the runtime environment
    --resource=t4 \ # Specify the resources to be used. By command bayes gear resource Can be in USAGE Fields see command
    --data hyperai/mnist/1:/input0 \ # The data to be bound
    -- python main.py # Entrance command

adopt HyperAI.yaml Can provide default commands for running tasks in the current directory. For example, we define a content as follows HyperAI.yaml:

data_bindings:
- data: hyperai/mnist/1
  path: /input0
resource: t4
env: tensorflow-1.12
command: "python main.py"

Simply enter the following command in the current directory to achieve the same task execution effect:

bayes gear run task

HyperAI.yaml Field description and specifications

HyperAI.yaml Currently divided into two parts:

1. Basic type parameters, include data_bindings resource env command parameters Five of them
2. Parameters for automatic parameter tuning. Included in hyper_tuning The specific content is in Automatic parameter tuning Introduction

Data_bindings

Refers to the bound data, support "Container output" as well as "data set" , Bind up to three at the same time. Divided into two parts: data and path

Data

data Refers to the bound data source. If the bound data source is a "Version of dataset" Its form is:

<userid>/<dataset-name>/<dataset-version>

For example, to bind HyperAI The following name is MNIST The first version of the dataset, his data The field is

hyperai/mnist/1

If the bound data source is a "Output of container" , Its form is:

<userid>/jobs/<job-id>/output

Like binding HyperAI Below test-project Under the container jfaqJeLMcPM The output. Its form is:

hyperai/jobs/jfaqJeLMcPM/output

Path

The other part is path Which directory in the container is specifically bound to the data source. The currently supported directories are as follows:

• /input0
• /input1
• /input2
• /output

be careful

catalogue /input0-2 For read-only path. The data source bound cannot be changed. But its binding speed is very fast./output For the working directory. Although it has read and write permissions. But bind the data to /output There will be additional data copy time in the directory. Additional storage capacity will also be generated when storing containers. Therefore, users need to decide the directory to bind in their own scenario.

A complete data_bindings The example is as follows:

data_bindings:
- data: hyperai/mnist/1
  path: /input0
- data: hyperai/jobs/jfaqJeLMcPM/output
  path: output

Resource

What computing power container is used. By command bayes gear resource You can see the supported types of computing power.

Env

What runtime environment is used. By command bayes gear env Can view supported runtime environments.

Command

Only when creating "Script execution" When needed. The entry command during task execution.

Parameters

Creating a task perhaps jupter During execution. Can be done through parameters Pass in a group key / value Formal parameters. The main functions of this parameter are as follows::

1. Convenient to record the important parameters of this execution, parameters It will be displayed in the execution interface

2. Pass custom parameters to execution. Simultaneously supporting two forms:

   • parameters The content will generate a file during initialization HyperAI_params.json Convenient program to read internal parameters through this file:

   • parameters The content will be appended to the entrance command in the form of command-line parameters, for example HyperAI.yaml The following content is included in it:

   ...
   command: python main.py
   parameters:
     input: /input0
     epochs: 5
   ...

   So the real execution entry command will be

   python main.py --input=/input0 --epochs=5

Overwrite the parameters in the configuration file with command-line arguments

meanwhile. You can also override the corresponding parameters through the command line. For example, regarding the previous text HyperAI.yaml The commands executed at the entrance can be overridden by the following commands:

bayes gear run task -- sleep 360

So the content of the submitted command will no longer be python main.py But rather sleep 360.