Using SCOPE

scope uses configuration files, generally written in the YAML syntax, to define what you wantt it do. This configuration is divided into seperate sections. Here, we give a brief overview. A complete reference is currently being written.

The first section, simply titled scope, defines general parameters for the program:

1
2
3
scope:
    couple_dir: "/path/to/directory/"
    number openMP processes: 8
In the example above, we define two things for the program.
couple_dir

Line 2

This entry defines where scope will save it’s files. These files generally include remap weights to be reused each time the coupler is called; gathered output files for processing for the other model; and intermediate files that may be interesting for diagnosis.

number openMP processes

Line 3

Since scope uses cdo in the background; you can use this to define how many processes you want to run cdo on. This generally speeds up regridding.

Next, there is a section which may optionally be defined, template_replacements. Here, you can store key/value pairs which will be replaced elsewhere in the configuration. As an example:

1
2
3
template_replacements:
    EXP_ID: "PI_1x10"
    DATE_PATTERN: "[0-9]{6}"

Now, any other time that {{ EXP_ID }} is used in the configuration, it will be replaced with PI_1x10. The syntax here is derived from the Jinja2 Python package used for templating.

Warning

I’m not sure what happens here if you try to use recursion in the template replacements!

You can also see in this section that you can define a DATE_PATTERN. Specific key/value pairs ending with the substring PATTERN are treated as a regular expression.

Next, you describe the models you wish to couple together.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
model_name:
    type: physical_domain
    griddes: built-in CDO grid description, or path to a SCRIP formatted file
    outdata_dir: /some/path
    code table: build-in CDO code table, or path to a file with GRB-style code table
    send:
        ...send directives...
    receive:
        ...receive directives...
    pre_step:
        ...description of pre step...
    post_step:
        ...description of post step...

In the generalised example above, we define:

model_name
A model to couple, in this case, model_name. Usually, this would be more specific, e.g. echam, openifs, pism, fesom.

Inside the model_name configuration, we again have:

type
This describes the type of the model; e.g. atmosphere, ice, ocean.
griddes
Here, you must specify which grid description to use for this model. This is the default for this model.
outdata_dir
This defines where scope will, by default, look for files for this particular model. However, you can override this on a case by case basis. See the send directives for more information.
code_table
Since scope is built on top of cdo, which supports grb files, here you can specify which code table to use in order to detect variable names when converting from grb to netcdf.
send
This configuration contains send directives for other coupling partners. More on this in the next section.
receive
This configuration is used to receive information from other models.
pre_step
Programs run before a particular step. Can be configured for each step separately, e.g. pre_preprocess, or pre_regrid.
post_step
Programs run after a particular step

Example: Configuration Files for SCOPE

A complete example configuration file is provided under examples/scope_config.yaml:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
template_replacements:
    EXP_ID: "PI_1x10"
    DATE_PATTERN: "[0-9]{6}"

scope:
    couple_dir: "/work/ollie/pgierz/scope_tests/couple/"
    number openMP processes: 8

echam:
    type: atmosphere
    griddes: T63
    outdata_dir: "/work/ollie/pgierz/scope_tests/outdata/echam/"
    code table: "echam6"
    pre_preprocess:
        program: "echo \"hello from pre_preprocess. Do you know: $(( 7 * 6 )) is the answer!\""
    send:
        ice:
            temp2:
                files:
                    pattern: "{{ EXP_ID }}_echam6_echam_{{ DATE_PATTERN }}.grb"
                    take:
                        newest: 12
                code table: "echam6"
            aprl:
                files:
                    dir: "/work/ollie/pgierz/scope_tests/outdata/echam/"
                    pattern: "{{ EXP_ID }}_echam6_echam_{{ DATE_PATTERN }}.grb"
                    take:
                        newest: 12
                code table: "/work/ollie/pgierz/scope_tests/outdata/echam/PI_1x10_185001.01_echam.codes"
            aprc:
                files:
                    dir: "/work/ollie/pgierz/scope_tests/outdata/echam/"
                    pattern: "{{ EXP_ID }}_echam6_echam_{{ DATE_PATTERN }}.grb"
                    take:
                        newest: 12

pism:
    type: ice
    griddes: ice.griddes
    recieve:
        atmosphere:
            temp2:
                interp: bil
                transformation:
                    - expr: "air_temp=temp2-273.15"
        ocean:
    send:
        atmosphere:
        ocean:

Example: PISM to ECHAM6

Command line interface

scope comes with a command line interface. For a very quick introduction:

$ scope --help

This will print usage information.

Any scope commands you normally would run in a batch job can also be individually targeted via command line arguments. In principle, the command structure is always the same, namely:

$ scope <command> ${CONFIG} ${WHOS_TURN}

This allows you to run one specific part of scope for a particular configuration assuming a particular model is currently doing something. As an example, this could take the form of:

$ scope preprocess ~/Code/scope/examples/scope_config.yaml echam

This would cause scope to run the prepare steps described for echam; in this particular case gathering output files, extracting variables, and placing the resulting file into the couple folder described in the configuration file. Note that also and pre- and post-processing hooks defined in the configuration file will also be run at this point.

All available commands are printed via scope --help.

Currently, the following commands are implemented:

  • preprocess
  • regrid

Python Library Usage

While the command line interface is nice for users who never want to actually touch scope code; we also support the ability to use scope functions in your own Python programs. this section describes how to use scope from a script.

To use scope in a project:

import scope

Consider having a look at the developer API for more detailed usage.