Quick Start

Starting of new project, based on VST Utils Framework, is rather simple. We recommend to create a virtual environment for each project to avoid conflicts in the system.

Let’s learn by example. All you need to do is run several commands. This manual consist of two parts:

Description of the process of creating a new application and the main commands for launching and deploying.
Description of the process of creating new entities in the application.

New application creation

Throughout this tutorial, we’ll go through a creation of a basic poll application.

Install VST Utils
pip install vstutils
In this case, we install a package with the minimum required set of dependencies for creating new projects. However, inside the project, the extra argument prod is used, which additionally installs the packages necessary for working in the production environment. There is also a set of test dependencies, which contains everything you need for testing and analyzing code coverage.

It is also worth noting extra dependencies as:
- rpc - install dependencies for asynchronous tasks working
- ldap - a set of dependencies for ldap authorization support
- doc - everything which needed to build documentation and to implement the delivery of documentation inside a running server
- pil - library for correct work of image validators
- boto3 - an additional set of packages for working with S3 storage outside of AWS
- sqs - a set of dependencies for connecting asynchronous tasks to SQS queues (can be used instead of the rpc).
You can combine multiple dependencies at the same time to assemble your set of functionality into a project. For example, to work an application with asynchronous tasks and media storage in MinIO, you will need the following command:
pip install vstutils[prod,rpc,boto3]
To install the most complete set of dependencies, you can use the common parameter all.
pip install vstutils[all]
Create new project, based on VST Utils
If this is your first time using vstutils, you’ll have to take care of some initial setup. Namely, you’ll need to auto-generate some code that establishes a vstutils application – a collection of settings for an instance of vstutils, including database configuration, Django-specific and vstutils-specific options and application-specific settings. To create new project execute following command:
python -m vstutils newproject --name {{app_name}}
This command will offer you to specify such options of new app, as:
- project name - name of your new application;
- project guiname - name of your new application, that will be used in GUI (web-interface);
- project directory - path to directory, where project will be created.
Or you can execute following command, that includes all needed data for new project creation.
python -m vstutils newproject --name {{app_name}} --dir {{app_dir}} --guiname {{app_guiname}} --noinput
This command creates new project without confirming any data.

These commands create several files in project directory.
/{{app_dir}}/{{app_name}} ├── frontend_src │ ├── app │ │ └── index │ ├── .editorconfig │ ├── .eslintrc.js │ └── .prettierrc ├── MANIFEST.in ├── package.json ├── README.rst ├── requirements-test.txt ├── requirements.txt ├── pyproject.toml ├── setup.py ├── {{app_name}} │ ├── __init__.py │ ├── __main__.py │ ├── models │ │ └── __init__.py │ ├── settings.ini │ ├── settings.py │ ├── web.ini │ └── wsgi.py ├── test.py ├── tox.ini └── webpack.config.jsdefault
where:
- frontend_src - directory that contains all sources for frontend;
- MANIFEST.in - this file is used for building installation package;
- {{app_name}} - directory with files of your application;
- package.json - this file contains list of frontend dependencies and commands to build;
- README.rst - default README file for your application (this file includes base commands for starting/stopping your application);
- requirements-test.txt - file with list of requirements for test environment;
- requirements.txt - file with list of requirements for your application;
- pyproject.toml - this file is used for building installation package;
- setup.py - this file is used for building installation package;
- test.py - this file is used for tests creation;
- tox.ini - this file is used for tests execution;
- webpack.config.js.default - this file contain minimal script for webpack (replace ‘.default’ if write smthg in ‘app.js’).
You should execute below commands from the /{{app_dir}}/{{app_name}}/ directory. It is good practice to use tox (should be installed before use) to create a debugging environment for your application. For these purposes, it is recommended to use tox -e contrib in the project directory, which will automatically create a new environment with the required dependencies.
Apply migrations
Let’s verify a newly created vstutils project does work. Change into the outer /{{app_dir}}/{{app_name}} directory, if you haven’t already, and run the following command:
python -m {{app_name}} migrate
This command create SQLite (by default) database with default SQL-schema. VSTUTILS supports all databases Django does.
Create superuser
python -m {{app_name}} createsuperuser
Start your application
python -m {{app_name}} web
Web-interface of your application has been started on the port 8080. You’ve started the vstutils production server based on uWSGI.

Warning

Now’s a good time to note: if you want to run the web-server with a debugger, then you should run the standard Django’s dev-server.

If you need to stop the server, use following command:
python -m {{app_name}} web stop=/tmp/{{app_name}}_web.pid

You’ve created the simplest application, based on VST Utils framework. This application only contains User Model. If you want to create your own models look at the section below.

Adding new models to application

If you want to add some new entities to your application, you need to do following on the back-end:

Create Model;
Create Serializer (optional);
Create View (optional);
Add created Model or View to the API;
Make migrations;
Apply migrations;
Restart your application.

Let’s look how you can do it on the AppExample - application, that has 2 custom models:

Task (abstraction for some tasks/activities, that user should do);
Stage (abstraction for some stages, that user should do to complete the task. This model is nested into the Task Model).

Models creation

Firstly, you need to create file {{model_name}}.py in the /{{app_dir}}/{{app_name}}/{{app_name}}/models directory.

Let make out an example from BModel:

class vstutils.models.BModel(*args, **kwargs)[source]

Default model class that generates model viewset, separate serializers for list() and retrieve(), filters, api endpoints and nested views.

Examples:

from django.db import models
from rest_framework.fields import ChoiceField
from vstutils.models import BModel

class Stage(BModel):
    name = models.CharField(max_length=256)
    order = models.IntegerField(default=0)

    class Meta:
        default_related_name = "stage"
        ordering = ('order', 'id',)
        # fields which would be showed on list.
        _list_fields = [
            'id',
            'name',
        ]
        # fields which would be showed on detail view and creation.
        _detail_fields = [
            'id',
            'name',
            'order'
        ]
        # make order as choices from 0 to 9
        _override_detail_fields = {
            'order': ChoiceField([str(i) for i in range(10)])
        }


class Task(BModel):
    name = models.CharField(max_length=256)
    stages = models.ManyToManyField(Stage)
    _translate_model = 'Task'

    class Meta:
        # fields which would be showed.
        _list_fields = [
            'id',
            'name',
        ]
        # create nested views from models
        _nested = {
            'stage': {
                'allow_append': False,
                'model': Stage
            }
        }

In this case, you create models which could converted to simple view, where:

POST/GET to /api/version/task/ - creates new or get list of tasks
PUT/PATCH/GET/DELETE to /api/version/task/:id/ - updates, retrieves or removes instance of task
POST/GET to /api/version/task/:id/stage/ - creates new or get list of stages in task
PUT/PATCH/GET/DELETE to /api/version/task/:id/stage/:stage_id - updates, retrieves or removes instance of stage in task.

To attach a view to an API insert the following code in settings.py:

API[VST_API_VERSION][r'task'] = {
    'model': 'your_application.models.Task'
}

For primary access to generated view inherit from Task.generated_view property.

To make translation on frontend easier use _translate_model attribute with model_name.

List of meta-attributes for generating a view:

_view_class - list of additional view classes to inherit from, class or string to import with base class ViewSet. Constants are also supported:
- read_only - to create a view only for viewing;
- list_only - to create a view with list only;
- history - to create a view only for viewing and deleting records.
CRUD-view is applied by default.
_serializer_class - class of API serializer; use this attribute to specify parent class for autogenerated serializers. Default is vstutils.api.serializers.VSTSerializer. Can take a string to import, serializer class or django.utils.functional.SimpleLazyObject.
_serializer_class_name - model name for OpenAPI definitions.This would be a model name in generated admin interface. Default is name of model class.
_list_fields or _detail_fields - list of fields which will be listed in entity list or detail view accordingly. Same as DRF serializers meta-attribute “fields”.
_override_list_fields or _override_detail_fields - mapping with names and field types that will be redeclared in serializer attributes(think of it as declaring fields in DRF ModelSerializer). Keep in mind that the fields specified here do not necessarily end up in the serializer class, but only if they are included in the corresponding lists.
_properties_groups - dict with key as group name and value as list of fields(str). Allows to group fields in sections on frontend.
_view_field_name - name of field frontend shows as main view name.
_non_bulk_methods - list of methods which must not used via bulk requests.
_extra_serializer_classes - mapping with additional serializers in viewset. For example, custom serializer, which will compute smth in action (mapping name). Value can be string for import. Important note: setting model attribute to None allows to use standard serializer generation mechanism and get fields from a list or detail serializer (set __inject_from__ serializer’s meta attribute to list or detail accordingly). In some cases, it is required to pass the model to the serializer. For these purposes, the constant LAZY_MODEL can be used as a meta attribute. Each time the serializer is used, the exact model where this serializer was declared will be set.
_filterset_fields - list/dict of filterset names for API-filtering. Default is list of fields in list view. During processing a list of fields checks for the presence of special field names and inherit additional parent classes. If the list contains id, class will inherit from vstutils.api.filters.DefaultIDFilter. If the list contains name, class will inherit from vstutils.api.filters.DefaultNameFilter. If both conditions are present, inheritance will be from all of the above classes. Possible values include list of fields to filter or dict where key is a field name and value is a Filter class. Dict extends attribute functionality and provides ability to override filter field class (None value disables overriding).
_search_fields - tuple or list of fields using for search requests. By default (or None) get all filterable fields in detail view.
_copy_attrs - list of model-instance attributes indicates that object is copyable with this attrs.
_nested - key-value mapping with nested views (key - nested name, kwargs for vstutils.api.decorators.nested_view decorator but supports model attribute as nested). model can be string for import. Use override_params when you need to override generated view parameters for nested view (works only with model as view).
_extra_view_attributes - key-value mapping with additional view attributes, but has less priority over generated attributes.
_hidden_on_frontend_list_fields - list with fields names which will be hidden in list view on frontend.
_hidden_on_frontend_detail_fields - list with fields names which will be hidden in detail view on frontend.
_detail_operations_availability_field_name - name of field which indicates availability of operations, field value can be dict where key is operation name and value is boolean flag or simply list of available operations.

In common, you can also add custom attributes to override or extend the default list of processing classes. Supported view attributes are filter_backends, permission_classes, authentication_classes, throttle_classes, renderer_classes and parser_classes. List of meta-attributes for settings of view is looks like:

_pre_{attribute} - List of classes included before defaults.
_{attribute} - List of classes included after defaults.
_override_{attribute} - boolean flag indicates that attribute override default viewset (otherwise appends). Default is False.

Note

You may need to create an action on generated view. Use vstutils.models.decorators.register_view_action decorator with the detail argument to determine applicability to a list or detail entry. In this case, the decorated method will take an instance of the view object as self attribute.

Note

In some cases, inheriting models may require to inherit Meta class from the base model. If the Meta is explicitly declared in the base class, then you can get it through the attribute OriginalMeta and use it for inheritance.

Note

Docstring of model will be reused for view descriptions. It is possible to write both a general description for all actions and description for each action using the following syntax:

General description for all actions.

action_name:
    Description for this action.

another_action:
    Description for another action.

The get_view_class() method is a utility method in the Django ORM model designed to facilitate the configuration and instantiation of Django Rest Framework (DRF) Generic ViewSets. It allows developers to define and customize various aspects of the associated DRF view class. The arguments passed to the function fully correspond to those specified earlier for the Meta class of the model, but without the underscore prefix.

Examples:

# Create simple list view with same fields
TaskViewSet = Task.get_view_class(view_class='list_only')

# Create view with overriding nested view params
from rest_framework.mixins import CreateModelMixin

TaskViewSet = Task.get_view_class(
    nested={
        "milestones": {
            "model": Stage,
            "override_params": {
                "view_class": ("history", CreateModelMixin)
            },
        },
    },
)

Developers can use this method to customize various aspects of the associated view, such as serializer classes, field configurations, filter backends, permission classes, etc. It uses attributes declared in meta attributes, but allows individual parts to be overriden.

An important aspect that distinguishes get_view_class() is the ignore_meta argument. It takes True when it’s necessary to completely ignore the values set in the model’s Meta class. This allows get_view_class() to generate a fully independent view.

More information about Models you can find in Django Models documentation.

If you don’t need to create custom serializers or view sets, you can go to this stage.

Serializers creation

Note - If you don’t need custom serializer you can skip this section

Firstly, you need to create file serializers.py in the /{{app_dir}}/{{app_name}}/{{app_name}}/ directory.

Then you need to add some code like this to serializers.py:

from datetime import datetime
from vstutils.api import serializers as vst_serializers
from . import models as models

class StageSerializer(models.Stage.generated_view.serializer_class):

    class Meta:
        model = models.Stage
        fields = ('id',
                'name',
                'order',)

    def update(self, instance, validated_data):
        # Put custom logic to serializer update
        instance.last_update = datetime.utcnow()
        super().update(instance, validated_data)

More information about Serializers you can find in Django REST Framework documentation for Serializers.

Views creation

Note - If you don’t need custom view set you can skip this section

Firstly, you need to create file views.py in the /{{app_dir}}/{{app_name}}/{{app_name}}/ directory.

Then you need to add some code like this to views.py:

from vstutils.api import decorators as deco
from vstutils.api.base import ModelViewSet
from . import serializers as sers
from .models import Stage, Task


class StageViewSet(Stage.generated_view):
    serializer_class_one = sers.StageSerializer


'''
Decorator, that allows to put one view into another
    * 'tasks' - suburl for nested view
    * 'methods=["get"]' - allowed methods for this view
    * 'manager_name='hosts' - Name of related QuerySet to the child model instances (we set it in HostGroup model as "hosts = models.ManyToManyField(Host)")
    *  'view=Task.generated_view' - Nested view, that will be child view for decorated view
'''
@nested_view('stage', view=StageViewSet)
class TaskViewSet(Task.generated_view):
    '''
    Task operations.
    '''

More information about Views and ViewSets you can find in Django REST Framework documentation for views.

Adding Models to API

To add created Models to the API you need to write something like this at the end of your settings.py file:

'''
Some code generated by VST Utils
'''

'''
Add Task view set to the API
Only 'root' (parent) views should be added there.
Nested views added automatically, that's why there is only Task view.
Stage view is added altogether with Task as nested view.
'''
API[VST_API_VERSION][r'task'] = {
    'view': 'newapp2.views.TaskViewSet'
}

'''
You can add model too.
All model generate base ViewSet with data that they have, if you don't create custom ViewSet or Serializer
'''
API[VST_API_VERSION][r'task'] = dict(
    model='newapp2.models.Task'
)

# Adds link to the task view to the GUI menu
PROJECT_GUI_MENU.insert(0, {
    'name': 'Task',
     # CSS class of font-awesome icon
    'span_class': 'fa fa-list-alt',
    'url': '/task'
})

Migrations creation

To make migrations open /{{app_dir}}/{{app_name}}/ directory and execute following command:

python -m {{app_name}} makemigrations {{app_name}}

More information about Migrations you can find in Django Migrations documentation.

Migrations applying

To apply migrations you need to open /{{app_dir}}/{{app_name}}/ directory and execute following command:

python -m {{app_name}} migrate

Restart of Application

To restart your application, firstly, you need to stop it (if it was started before):

python -m {{app_name}} web stop=/tmp/{{app_name}}_web.pid

And then start it again:

python -m {{app_name}} web

After cache reloading you will see following page:

As you can see, link to new Task View has been added to the sidebar menu. Let’s click on it.

_images/app_example_task_empty_list_page.png

There is no task instance in your app. Add it using ‘new’ button.

After creating a new task you’ll see a following page:

_images/app_example_created_task_page.png

As you can see, there is ‘stages’ button, that opens page with this task’s stages list. Let’s click on it.

_images/app_example_stage_empty_list_page.png

There is no stage instances in your app. Let’s create 2 new stages.

After stages creation page with stages list will looks like this:

Sorting by ‘order’ field works, as we mentioned in the our models.py file for Stage Model.

Additional information about Django and Django REST Framework you can find in Django documentation and Django REST Framework documentation.