Usage guide

This library allows to interact with packages supported by SIO2 system. It’s used in two important components of SIO2:

  • OIOIOI, which is the web interface for SIO2 system,

  • sinol-make, which is the local tool for testing packages.

The purpose of this library is to keep the code for package management in one place, so that it can be reused in both components. This will also allow developers to write their own tools that can interact with SIO2 packages.

Supported package types

Currently, this library supports the following package types:

  • sinolpack – a package format originally used by SIO2 system, which is a simple archive format with metadata, and is used for most packages in SIO2 system.

Next planned package types to be supported are:

  • sinol3pack – a new package format that is being developed for SIO2/SIO3 system, which will be a more advanced format with better support for metadata and dependencies,

  • Codeforces packages – a package format used by Codeforces system, which is a popular competitive programming platform.

Installation

To install the library, you can use pip. There are two ways to install it, depending on the way you want to use it:

  • if you want to use the library locally, without the need for Django, you can install it with the following command:

    pip install sio3pack
    
  • if you want to use the library with Django support, for example when developing OIOIOI, you can install it with the following command:

    pip install sio3pack[django]
    

Initializing without Django support

When using SIO3Pack without Django support, there is only one way to initialize a package, which is by using the sio3pack.from_file() function. This function takes a file path to a directory with the package or a archived package file, and returns an instance of sio3pack.packages.package.Package class, which can be used to interact with the package.

This class is an abstract base class, so for different package types, you will receive different subclasses of this class. For example for sinolpack packages, you will get an instance of sio3pack.packages.sinolpack.Sinolpack class.

Example when importing a sinolpack package:

import sio3pack
from sio3pack.packages.sinolpack import Sinolpack


package: Sinolpack = sio3pack.from_file("path/to/sinolpack/package")

print(package.full_name)  # Prints the full title of the package

The sio3pack.from_file() function can also take an optional argument configuration, which is an instance of sio3pack.packages.package.SIO3PackConfig class. For local usage, this class mainly provides available compilers. It is possible to automatically detect the configuration by using the sio3pack.packages.package.SIO3PackConfig.detect() method, which will try to detect the configuration based on the current environment. Example usage:

import sio3pack
from sio3pack.packages.sinolpack import Sinolpack
from sio3pack.packages.package import SIO3PackConfig

configuration = SIO3PackConfig.detect()
package: Sinolpack = sio3pack.from_file("path/to/sinolpack/package", configuration=configuration)

print(package.full_name)  # Prints the full title of the package

Initializing with Django support

When using SIO3Pack with Django support, you can initialize a package using the same sio3pack.from_file() function, but you can also use the sio3pack.from_db() function, which allows you to initialize a package from the database.

After initializing the package from file, you can use the sio3pack.packages.package.Package.save_to_db() method to save the package to the database. This method will create a new package in the database, or update an existing one if it already exists.

The sio3pack.packages.package.SIO3PackConfig class can also take the Django settings, which is useful when you want to use the library with Django support.

The important distinction between a package from file and a package from the database is that the package from database is lazy-loaded, meaning that the metadata and files are not loaded until they are accessed. This is useful for saving memory and improving performance, especially for OIOIOI.

Example usage:

import sio3pack
from sio3pack.packages.sinolpack import Sinolpack
from sio3pack.packages.package import SIO3PackConfig

from django.conf import settings


package: Sinolpack = sio3pack.from_file("path/to/sinolpack/package")
problem_id = 1  # The `save_to_db` functions requires a problem ID to save the package to the database
package.save_to_db(problem_id)


configuration = SIO3PackConfig.detect()
configuration.django_settings = settings  # Set the Django settings for the configuration
from_db: Sinolpack = sio3pack.from_db(problem_id, configuration=configuration)
print(from_db.full_name)  # Prints the full title of the package from the database

Interacting with workflows

The most important part of the new packages specification are the workflows, which are a set of steps that can be executed by the sio3workers. There are default workflows for all types of packages, but you can also create your own workflows by creating a workflows.json file in the package directory. For more information, read the workflows section of the documentation.

There are a couple of important workflows that can be used:

  • unpack workflow – for the first time unpacking the package, which will generate tests and verify them

  • run workflow – for running a program on the tests from the package

  • generate user out workflow – for generating an output of user’s program on a test from the package

  • test run workflow – for generating an output for user’s program on a user-provided test

Each method for generating workflows returns an instance of sio3pack.workflow.WorkflowOperation class, which can be used to get the workflows. This class has a method sio3pack.workflow.WorkflowOperation.get_workflow(), which yields instances of sio3pack.workflow.Workflow class until there are workflows to be run. The sio3pack.workflow.Workflow class can be passed to a sio3worker executor, which will then run the workflow and return the results. After each workflow run, the results should be passed to sio3pack.workflow.WorkflowOperation.return_results() method. Example usage of a workflow operation:

import sio3pack, sio3worker
from sio3pack.packages.sinolpack import Sinolpack
from sio3pack.workflow import WorkflowOperation

package: Sinolpack = sio3pack.from_file("path/to/sinolpack/package")
wf_op: WorkflowOperation = package.get_some_workflow_operation()

for workflow in wf_op.get_workflow():
  results = sio3worker.run_workflow(workflow)  # This will run the workflow using the sio3worker executor
  wf_op.return_results(results)  # This will return the results of the workflow to the operation

Unpack workflow

This workflow should be used after initializing a package from file. It will generate tests, verify them and save the information about the tests on the package. This workflow uses these sub-workflows:

  • ingen – for generating input tests

  • outgen – for generating output tests

  • inwer – for verifying the input tests

These workflows can be created by calling the sio3pack.packages.package.Package.get_unpack_operation() method on the package instance. This method will return the workflow operation. Example usage:

import sio3pack, sio3worker
from sio3pack.packages.sinolpack import Sinolpack
from sio3pack.workflow import WorkflowOperation

package: Sinolpack = sio3pack.from_file("path/to/sinolpack/package")
unpack_op: WorkflowOperation = package.get_unpack_operation()

for workflow in unpack_op.get_workflow():
  results = sio3worker.run_workflow(workflow)  # This will run the workflow using the sio3worker executor
  unpack_op.return_results(results)  # This will return the results of the workflow to the operation

# After unpacking, you can save the package to the database
package.save_to_db(problem_id)

Run workflow

This workflow is used for running a program on the tests from the package. It can be created by calling the sio3pack.packages.package.Package.get_run_operation() method on the package instance. This method requires a program argument, which is an instance of sio3pack.files.File class, which represents the program to be run. It also has an optional argument tests, which allows to specify a list of tests to be run. If not specified, all tests from the package will be used. Example usage:

import sio3pack, sio3worker
from sio3pack.packages.sinolpack import Sinolpack
from sio3pack.files import File
from sio3pack.workflow import WorkflowOperation

package: Sinolpack = sio3pack.from_file("path/to/sinolpack/package")
program: File = package.get_program("main")  # Get the main program file from the package
run_op: WorkflowOperation = package.get_run_operation(program)

for workflow in run_op.get_workflow():
  results = sio3worker.run_workflow(workflow)  # This will run the workflow using the sio3worker executor
  run_op.return_results(results)  # This will return the results of the workflow to the operation

User out workflow

This workflow is used for generating an output of user’s program on a test from the package. It can be created by calling the sio3pack.packages.package.Package.get_user_out_operation() method on the package instance. This method requires two arguments: program and test. The program argument is an instance of sio3pack.files.File class, which represents the user’s program to be run, and the test argument is an instance of sio3pack.test.Test class, which represents the test to be run. Example usage:

import sio3pack, sio3worker
from sio3pack.packages.sinolpack import Sinolpack
from sio3pack.files import File
from sio3pack.test import Test
from sio3pack.workflow import WorkflowOperation

package: Sinolpack = sio3pack.from_file("path/to/sinolpack/package")
program: File = package.get_program("main")  # Get the main program file from the package
test: Test = package.tests[0]  # Get the first test from the package
user_out_op: WorkflowOperation = package.get_user_out_operation(program, test)

for workflow in user_out_op.get_workflow():
  results = sio3worker.run_workflow(workflow)  # This will run the workflow using the sio3worker executor
  user_out_op.return_results(results)  # This will return the results of the workflow to the operation

Test run workflow

This workflow is used for generating an output for user’s program on a user-provided test. It can be created by calling the sio3pack.packages.package.Package.get_test_run_operation() method on the package instance. This method requires two arguments: program and test. Both arguments are instances of sio3pack.files.File class, where program represents the user’s program to be run, and test represents the user-provided test to be run. Example usage:

import sio3pack, sio3worker
from sio3pack.packages.sinolpack import Sinolpack
from sio3pack.files import LocalFile
from sio3pack.workflow import WorkflowOperation

package: Sinolpack = sio3pack.from_file("path/to/sinolpack/package")
program: File = package.get_program("main")  # Get the main program file from the package
user_test: LocalFile = LocalFile("path/to/user/test")  # User-provided test file
test_run_op: WorkflowOperation = package.get_test_run_operation(program, user_test)

for workflow in test_run_op.get_workflow():
  results = sio3worker.run_workflow(workflow)  # This will run the workflow using the sio3worker executor
  test_run_op.return_results(results)  # This will return the results of the workflow to the operation

More information

For more information on how to use the library, you can check the documentation for the specific package types, such as sio3pack.packages.sinolpack.Sinolpack. You can also check the workflows section for examples of how to implement your own workflows using the library.