Backend API manual

VST Utils framework uses Django, Django Rest Framework, drf-yasg and Celery.

Models

A model is the single, definitive source of truth about your data. It contains essential fields and behavior for the data you’re storing. Usually best practice is to avoid writing views and serializers manually, as BModel provides plenty of Meta attributes to autogenerate serializers and views for many use cases.

Default Django model classes overrides in vstutils.models module.

class vstutils.models.BModel(*args, **kwargs)[source]

Default model class that generates model viewset, separate serializers for list() and retrieve(), filters, api endpoints and nested views.

Examples:
from django.db import models
from rest_framework.fields import ChoiceField
from vstutils.models import BModel

class Stage(BModel):
    name = models.CharField(max_length=256)
    order = models.IntegerField(default=0)

    class Meta:
        default_related_name = "stage"
        ordering = ('order', 'id',)
        # fields which would be showed on list.
        _list_fields = [
            'id',
            'name',
        ]
        # fields which would be showed on detail view and creation.
        _detail_fields = [
            'id',
            'name',
            'order'
        ]
        # make order as choices from 0 to 9
        _override_detail_fields = {
            'order': ChoiceField((str(i) for i in range(10)))
        }


class Task(BModel):
    name = models.CharField(max_length=256)
    stages = models.ManyToManyField(Stage)
    _translate_model = 'Task'

    class Meta:
        # fields which would be showed.
        _list_fields = [
            'id',
            'name',
        ]
        # create nested views from models
        _nested = {
            'stage': {
                'allow_append': False,
                'model': Stage
            }
        }

In this case, you create models which could converted to simple view, where:

  • POST/GET to /api/version/task/ - creates new or get list of tasks

  • PUT/PATCH/GET/DELETE to /api/version/task/:id/ - updates, retrieves or removes instance of task

  • POST/GET to /api/version/task/:id/stage/ - creates new or get list of stages in task

  • PUT/PATCH/GET/DELETE to /api/version/task/:id/stage/:stage_id - updates, retrieves or removes instance of stage in task.

To attach a view to an API insert the following code in settings.py:

API[VST_API_VERSION][r'task'] = {
    'model': 'your_application.models.Task'
}

For primary access to generated view inherit from Task.generated_view property.

To make translation on frontend easier use _translate_model attribute with model_name

List of meta-attributes for generating a view:

  • _view_class - list of additional view classes to inherit from, class or string to import with base class ViewSet. Constants are also supported:

    • read_only - to create a view only for viewing;

    • list_only - to create a view with list only;

    • history - to create a view only for viewing and deleting records.

    CRUD-view is applied by default.

  • _serializer_class - class of API serializer; use this attribute to specify parent class for autogenerated serializers. Default is vstutils.api.serializers.VSTSerializer. Can take a string to import, serializer class or django.utils.functional.SimpleLazyObject.

  • _serializer_class_name - model name for OpenAPI definitions.This would be a model name in generated admin interface. Default is name of model class.

  • _list_fields or _detail_fields - list of fields which will be listed in entity list or detail view accordingly. Same as DRF serializers meta-attribute “fields”.

  • _override_list_fields or _override_detail_fields - mapping with names and field types that will be redeclared in serializer attributes(think of it as declaring fields in DRF ModelSerializer).

  • _properties_groups - dict with key as group name and value as list of fields(str). Allows to group fields in sections on frontend.

  • _view_field_name - name of field frontend shows as main view name.

  • _non_bulk_methods - list of methods which must not used via bulk requests.

  • _extra_serializer_classes - mapping with additional serializers in viewset. For example, custom serializer, which will compute smth in action (mapping name). Value can be string for import. Important note: setting model attribute to None allows to use standard serializer generation mechanism and get fields from a list or detail serializer (set __inject_from__ serializer’s meta attribute to list or detail accordingly). In some cases, it is required to pass the model to the serializer. For these purposes, the constant LAZY_MODEL can be used as a meta attribute. Each time the serializer is used, the exact model where this serializer was declared will be set.

  • _filterset_fields - list/dict of filterset names for API-filtering. Default is list of fields in list view. During processing a list of fields checks for the presence of special field names and inherit additional parent classes. If the list contains id, class will inherit from vstutils.api.filters.DefaultIDFilter. If the list contains name, class will inherit from vstutils.api.filters.DefaultNameFilter. If both conditions are present, inheritance will be from all of the above classes. Possible values include list of fields to filter or dict where key is a field name and value is a Filter class. Dict extends attribute functionality and provides ability to override filter field class (None value disables overriding).

  • _search_fields - tuple or list of fields using for search requests. By default (or None) get all filterable fields in detail view.

  • _copy_attrs - list of model-instance attributes indicates that object is copiable with this attrs.

  • _nested - key-value mapping with nested views (key - nested name, kwargs for vstutils.api.decorators.nested_view decorator but supports model attribute as nested). model can be string for import.

  • _extra_view_attributes - key-value mapping with additional view attributes, but has less priority over generated attributes.

In common, you can also add custom attributes to override or extend the default list of processing classes. Supported view attributes are filter_backends, permission_classes, authentication_classes, throttle_classes, renderer_classes and parser_classes. List of meta-attributes for settings of view is looks like:

  • _pre_{attribute} - List of classes included before defaults.

  • _{attribute} - List of classes included after defaults.

  • _override_{attribute} - boolean flag indicates that attribute override default viewset (otherwise appends). Default is False.

Note

You may need to create an action on generated view. Use vstutils.models.decorators.register_view_action decorator with the detail argument to determine applicability to a list or detail entry. In this case, the decorated method will take an instance of the view object as self attribute.

Note

In some cases, inheriting models may require to inherit Meta class from the base model. If the Meta is explicitly declared in the base class, then you can get it through the attribute OriginalMeta and use it for inheritance.

Note

Docstring of model will be reused for view descriptions. It is possible to write both a general description for all actions and description for each action using the following syntax:

General description for all actions.

action_name:
    Description for this action.

another_action:
    Description for another action.
hidden

If hidden is set to True, entry will be excluded from query in BQuerySet.

id

Primary field for select and search in API.

class vstutils.models.Manager(*args, **kwargs)[source]

Default VSTUtils manager. Used by BaseModel and BModel. Uses BQuerySet as base.

class vstutils.models.queryset.BQuerySet(model=None, query=None, using=None, hints=None)[source]

Represent a lazy database lookup for a set of objects. Allows to override default iterable class by custom_iterable_class attribute (class with __iter__ method which returns generator of model objects) and default query class by custom_query_class attribute (class inherited from django.db.models.sql.query.Query).

cleared()[source]

Filter queryset for models with attribute ‘hidden’ and exclude all hidden objects.

get_paginator(*args, **kwargs)[source]

Returns initialized object of vstutils.utils.Paginator over current instance’s QuerySet. All args and kwargs go to to Paginator’s constructor.

paged(*args, **kwargs)[source]

Returns paginated data with custom Paginator-class. Uses PAGE_LIMIT from global settings by default.

class vstutils.models.decorators.register_view_action(*args, **kwargs)[source]

Decorator for turning model methods to generated view actions. The decorated method becomes a method of generated view and self is an view object. See supported args in vstutils.api.decorators.subaction().

Note

Sometimes you may need to use proxy models with a common set of actions. To receive the action by the proxy model, pass the named argument inherit with True value.

Note

Often, an action does not transfer any parameters and requires only sending an empty query. To speed up development, we set the default serializer to vstutils.api.serializers.EmptySerializer.

You can also use custom models without using database:

class vstutils.custom_model.FileModel(*args, **kwargs)[source]

Custom model which loads data from YAML-file instead of database. Path to the file stored in FileModel.file_path attribute.

Examples:

Source file stored in /etc/authors.yaml with content:

- name: "Sergey Klyuykov"
- name: "Michael Taran"

Example:

from vstutils.custom_model import FileModel, CharField


class Authors(FileModel):
    name = CharField(max_length=512)

    file_path = '/etc/authors.yaml'
class vstutils.custom_model.ListModel(*args, **kwargs)[source]

Custom model which uses a list of dicts with data (attribute ListModel.data) instead of database records. Useful when you have a simple list of data.

Examples:
from vstutils.custom_model import ListModel, CharField


class Authors(ListModel):
    name = CharField(max_length=512)

    data = [
        {"name": "Sergey Klyuykov"},
        {"name": "Michael Taran"},
    ]

Sometimes, it may be necessary to switch the data source. For these purposes, you should use the setup_custom_queryset_kwargs function, which takes various named arguments, which are also passed to the data initialization function. One such argument for ListModel is date_source, which takes any iterable object.

Examples:
from vstutils.custom_model import ListModel, CharField


class Authors(ListModel):
    name = CharField(max_length=512)

qs = Authors.objects.setup_custom_queryset_kwargs(data_source=[
    {"name": "Sergey Klyuykov"},
    {"name": "Michael Taran"},
])

In this case, we setup source list via setup_custom_queryset_kwargs function, and any other chained call is going to work with this data.

data = []

List with data dicts. Empty by default.

Model Fields

class vstutils.models.fields.FkModelField(to, on_delete, related_name=None, related_query_name=None, limit_choices_to=None, parent_link=False, to_field=None, db_constraint=True, **kwargs)[source]

Extends django.db.models.ForeignKey. Use this field in vstutils.models.BModel to get vstutils.api.FkModelField in serializer. To set Foreign Key relation set to argument to string path to model or to Model Class as in django.db.models.ForeignKey

class vstutils.models.fields.MultipleFieldFile(instance, field, name)[source]

Subclasses django.db.models.fields.files.FieldFile. Provides MultipleFieldFile.save() and MultipleFieldFile.delete() to manipulate the underlying file, as well as update the associated model instance.

delete(save=True)[source]

Delete file from storage and from object attr.

save(name, content, save=True)[source]

Save changes in file to storage and to object attr.

class vstutils.models.fields.MultipleFileDescriptor(field)[source]

Subclasses django.db.models.fields.files.FileDescriptor to handle list of files. Return a list of MultipleFieldFile when accessed so you can write code like:

from myapp.models import MyModel
instance = MyModel.objects.get(pk=1)
instance.files[0].size
get_file(file, instance)[source]

Always return valid attr_class object.For details on logic see django.db.models.fields.files.FileDescriptor.__get__().

class vstutils.models.fields.MultipleFileField(**kwargs)[source]

Subclasses django.db.models.fields.files.FileField. Field for storing a list of Storage-kept files. All args passed to FileField.

attr_class

alias of MultipleFieldFile

descriptor_class

alias of MultipleFileDescriptor

class vstutils.models.fields.MultipleFileMixin(**kwargs)[source]

Mixin suited to use with django.db.models.fields.files.FieldFile to transform it to a Field with list of files.

get_prep_value(value)[source]

Prepare value for database insertion

pre_save(model_instance, add)[source]

Call .save() method on every file in list

class vstutils.models.fields.MultipleImageField(**kwargs)[source]

Field for storing a list of storage-kept images. All args are passed to django.db.models.fields.files.ImageField, except height_field and width_field, they are not currently implemented.

attr_class

alias of MultipleImageFieldFile

descriptor_class

alias of MultipleFileDescriptor

class vstutils.models.fields.MultipleImageFieldFile(instance, field, name)[source]

Subclasses MultipleFieldFile and ImageFile mixin, handles deleting _dimensions_cache when file is deleted.

class vstutils.models.fields.MultipleNamedBinaryFileInJSONField(*args, db_collation=None, **kwargs)[source]

Extends django.db.models.TextField. Use this field in vstutils.models.BModel to get vstutils.api.MultipleNamedBinaryFileInJSONField in serializer.

class vstutils.models.fields.MultipleNamedBinaryImageInJSONField(*args, db_collation=None, **kwargs)[source]

Extends django.db.models.TextField. Use this field in vstutils.models.BModel to get vstutils.api.MultipleNamedBinaryImageInJSONField in serializer.

class vstutils.models.fields.NamedBinaryFileInJSONField(*args, db_collation=None, **kwargs)[source]

Extends django.db.models.TextField. Use this field in vstutils.models.BModel to get vstutils.api.NamedBinaryFileInJSONField in serializer.

class vstutils.models.fields.NamedBinaryImageInJSONField(*args, db_collation=None, **kwargs)[source]

Extends django.db.models.TextField. Use this field in vstutils.models.BModel to get vstutils.api.NamedBinaryImageInJSONField in serializer.

Web API

Web API is based on Django Rest Framework with additional nested functions.

Fields

The Framework includes a list of convenient serializer fields. Some of them take effect only in generated admin interface.

Additional serializer fields for generating OpenAPI and GUI.

class vstutils.api.fields.AutoCompletionField(*args, **kwargs)[source]

Field that provides autocompletion on frontend, using specified list of objects.

Parameters
  • autocomplete (list,tuple,str) – Autocompletion reference. You can set list/tuple with values or set OpenApi schema definition name. For definition name GUI will find optimal link and will show values based on autocomplete_property and autocomplete_represent arguments.

  • autocomplete_property (str) – this argument indicates which attribute will be get from OpenApi schema definition model as value.

  • autocomplete_represent – this argument indicates which attribute will be get from OpenApi schema definition model as represent value.

  • use_prefetch (bool) – prefetch values on frontend at list-view. Default is True.

Note

Effective only in GUI. Works similar to VSTCharField in API.

class vstutils.api.fields.BinFileInStringField(*args, **kwargs)[source]

Field extends FileInStringField, but work with binary(base64) files.

Parameters

media_types (tuple,list) – List of MIME types to select on the user’s side. Supported syntax using *. Default: [‘*/*’]

Note

Effective only in GUI. Works similar to VSTCharField in API.

class vstutils.api.fields.CSVFileField(*args, **kwargs)[source]

Field extends FileInStringField, using for works with csv files. This field provides the display of the loaded data in the form of a table.

Parameters
  • items (Serializer) –

    The config of the table. This is a drf or vst serializer which includes char fields which are the keys in the dictionaries into which the data from csv is serialized and the names for columns in a table. The fields must be in the order you want them to appear in the table. Following options may be included:

    • label: human readable column name

    • required: Defines whether the field should be required. False by default.

  • min_column_width (int) – Minimum cell width. Default is 200 px.

  • delimiter (str) – The delimiting character.

  • lineterminator (str) – The newline sequence. Leave blank to auto-detect. Must be one of \r, \n, or \r\n.

  • quotechar (str) – The character used to quote fields.

  • escapechar (str) – The character used to escape the quote character within a field.

  • media_types (tuple,list) – List of MIME types to select on the user’s side. Supported syntax using *. Default: ['text/csv']

class vstutils.api.fields.CommaMultiSelect(*args, **kwargs)[source]

Field containing a list of values with specified separator(default: “,”). Gets list of values from another model or custom list. Provides autocompletion as AutoCompletionField, but with comma-lists. Suited for property-fields in model where main logic is already implemented or with CharField.

Parameters
  • select (str,tuple,list) – OpenApi schema definition name or list with values.

  • select_separator (str) – separator of values. Default is comma.

  • select_property,select_represent – work as autocomplete_property and autocomplete_represent. Default is name.

  • use_prefetch – prefetch values on frontend at list-view. Default is False.

  • make_link – Show value as link to model. Default is True.

  • dependence (dict) – Dictionary, where keys are name of field from the same model, and values are name of query filter .If at least one of the fields that we depend on is non nullable, required and set to null, autocompletion list will be empty and field will be disabled.

Note

Effective only in GUI. Works similar to VSTCharField in API.

class vstutils.api.fields.DeepFkField(only_last_child=False, parent_field_name='parent', **kwargs)[source]

Extends FkModelField, but displays as tree on frontend.

Warning

This field does not support dependence. Use filters at your own risk, as it would rather break the tree structure.

Parameters
  • only_last_child (bool) – if True then only allows a value to be selected if it has no children. Default is False

  • parent_field_name (str) – name of parent field in model. Default is parent

class vstutils.api.fields.DependEnumField(*args, **kwargs)[source]

Field extends DynamicJsonTypeField but its value is not transformed to json and would be given as is. Useful for property in models or for actions.

Parameters
  • field (str) – field in model which value change will change type of current value.

  • types – key-value mapping where key is value of subscribed field and value is type (in OpenApi format) of current field.

  • choices (dict) – variants of choices for different subscribed field values. Uses mapping where key is value of subscribed field and value is list with values to choice.

Note

Effective only in GUI. In API works similar to VSTCharField without value modification.

class vstutils.api.fields.DependFromFkField(*args, **kwargs)[source]

Field extends DynamicJsonTypeField. Validates field data by field_attribute chosen in related model. By default, any value of field_attribute validates as VSTCharField. To override this behavior set dict attribute {field_attribute value}_fields_mapping in related model where:

  • key - string representation of value type which is received from related instance field_attribute.

  • value - rest_framework.Field instance for validation.

Parameters
  • field (str) – field in model which value change changes type of current value. Field must be FkModelField.

  • field_attribute (str) – attribute of model related model instance with name of type.

Warning

field_attribute in related model must be rest_framework.ChoicesField or GUI will show field as simple text.

class vstutils.api.fields.DynamicJsonTypeField(*args, **kwargs)[source]

Field which type is based on another field. It converts value to internal string and represent field as json object.

Parameters
  • field (str) – field in model which value change will change type of current value.

  • types – key-value mapping where key is value of subscribed field and value is type (in OpenApi format) of current field.

  • choices (dict) – variants of choices for different subscribed field values. Uses mapping where key is value of subscribed field and value is list with values to choice.

Note

Effective only in GUI. In API works similar to VSTCharField without value modifications.

class vstutils.api.fields.FileInStringField(*args, **kwargs)[source]

Field extends VSTCharField and saves file’s content as string.

Value must be text (not binary) and saves in model as is.

Parameters

media_types (tuple,list) – List of MIME types to select on the user’s side. Supported syntax using *. Default: ['*/*']

Note

Take effect only in GUI. In API it would behave as VSTCharField.

class vstutils.api.fields.FkField(*args, **kwargs)[source]

Implementation of ForeignKeyField.You can specify which field of a related model will be stored in field(default: “id”), and which will represent field on frontend.

Parameters
  • select (str) – OpenApi schema definition name.

  • autocomplete_property (str) – this argument indicates which attribute will be get from OpenApi schema definition model as value. Default is id.

  • autocomplete_represent – this argument indicates which attribute will be get from OpenApi schema definition model as represent value. Default is name.

  • use_prefetch (bool) – prefetch values on frontend at list-view. Default is True.

  • make_link (bool) – show value as link to model. Default is True.

  • dependence (dict) –

    dictionary, where keys are names of a field from the same model, and keys are name of query filter. If at least one of the fields that we depend on is non nullable, required and set to null, autocompletion list will be empty and field will be disabled.

    There are some special keys for dependence dictionary to get data that is stored on frontend without additional database query:

    '<<pk>>', - get primary key of current instance

    '<<view_name>>' get view name from Vue component,

    '<<parent_view_name>>' get view name from Vue component,

    '<<view_level>>' get view level,

    '<<operation_id>>' get operation_id,

    '<<parent_operation_id'>> get parent_operation_id

Examples:
field = FkField(select=Category, dependence={'<<pk>>': 'my_filter'})

This filter will get pk of current object and make query on frontend ‘/category?my_filter=3’ where ‘3’ is primary key of current instance.

Parameters

filters (dict) – dictionary, where keys are names of a field from a related (by this FkField) model, and values are values of that field.

Note

Intersection of dependence.values() and filters.keys() will throw error to prevent ambiguous filtering.

Note

Effective only in GUI. Works similar to rest_framework.IntegerField in API.

class vstutils.api.fields.FkModelField(*args, **kwargs)[source]

Extends FkField, but stores referred model class. This field is useful for django.db.models.ForeignKey fields in model to set.

Parameters
  • select (vstutils.models.BModel,vstutils.api.serializers.VSTSerializer) – model class (based on vstutils.models.BModel) or serializer class which used in API and has path in OpenApi schema.

  • autocomplete_property (str) – this argument indicates which attribute will be get from OpenApi schema definition model as value. Default is id.

  • autocomplete_represent – this argument indicates which attribute will be get from OpenApi schema definition model as represent value. Default is name.

  • use_prefetch – prefetch values on frontend at list-view. Default is True.

  • make_link – Show value as link to model. Default is True.

Warning

Model class get object from database during .to_internal_value execution. Be careful on mass save executions.

Warning

Permissions to model which is referred by this field, are not to be checked. You should check it manually in signals or validators.

class vstutils.api.fields.HtmlField(*args, **kwargs)[source]

Field contains html text and marked as format:html. The field does not validate whether its content is HTML.

Warning

To avoid vulnerability, do not allow users to modify this data because users ate able to execute their scripts.

Note

Effective only in GUI. Works similar to VSTCharField in API.

class vstutils.api.fields.MaskedField(*args, **kwargs)[source]

Extends class ‘rest_framework.serializers.CharField’. Field that applies mask to value

Parameters

mask (dict, str) – IMask

Note

Effective only on frontend.

class vstutils.api.fields.MultipleNamedBinaryFileInJsonField(*args, **kwargs)[source]

Extends NamedBinaryFileInJsonField but uses list of JSONs. Allows to operate with multiple files as list of NamedBinaryFileInJsonField.

Attrs: NamedBinaryInJsonField.file: if True, accept only subclasses of File as input. If False, accept only string input. Default: False.

file_field

alias of MultipleFieldFile

class vstutils.api.fields.MultipleNamedBinaryImageInJsonField(*args, **kwargs)[source]

Extends MultipleNamedBinaryFileInJsonField but uses list of JSONs. Used for operating with multiple images and works as list of NamedBinaryImageInJsonField.

class vstutils.api.fields.NamedBinaryFileInJsonField(*args, **kwargs)[source]

Field that takes JSON with properties: * name - string - name of file; * mediaType - string - MIME type of file * content - base64 string - content of file.

This field is useful for saving binary files with their names in django.db.models.CharField or django.db.models.TextField model fields. All manipulations with decoding and encoding binary content data executes on client. This imposes reasonable limits on file size.

Additionally, this field can construct django.core.files.uploadedfile.SimpleUploadedFile from incoming JSON and store it as file in django.db.models.FileField if file argument is set to True

Attrs: NamedBinaryInJsonField.file: if True, accept only subclasses of File as input. If False, accept only string input. Default: False.

Note

Effective only in GUI. Works similar to VSTCharField in API.

class vstutils.api.fields.NamedBinaryImageInJsonField(*args, **kwargs)[source]

Extends NamedBinaryFileInJsonField to represent image on frontend (if binary image is valid).Validate this field with vstutils.api.validators.ImageValidator.

class vstutils.api.fields.PasswordField(*args, **kwargs)[source]

Extends CharField but in schema set format to password. Show all characters as asterisks instead of real value in GUI.

class vstutils.api.fields.PhoneField(*args, **kwargs)[source]

Extends class ‘rest_framework.serializers.CharField’. Field for for phone in international format

class vstutils.api.fields.RatingField(min_value=0, max_value=5, step=1, front_style='stars', **kwargs)[source]

Extends class ‘rest_framework.serializers.FloatField’. This field represents a rating form input on frontend. Grading limits can be specified with ‘min_value=’ and ‘max_value=’, defaults are 0 to 5.Minimal step between grades are specified in ‘step=’, default - 1.Frontend visual representation can be chosen with ‘front_style=’, available variants are listed in ‘self.valid_front_styles’.

for ‘slider’ front style, you can specify slider color, by passing valid color to ‘color=’. for ‘fa_icon’ front style, you can specify FontAwesome icon that would be used for displaying rating, by passing a valid FontAwesome icon code to ‘fa_class=’.

Parameters
  • min_value (float, int) – minimal level

  • max_value (float, int) – maximal level

  • step (float, int) – minimal step between levels

  • front_style (str) – visualization on frontend field. Allowed: [‘stars’, ‘slider’, ‘fa_icon’].

  • color (str) – color of rating element (star, icon or slider) in css format

  • fa_class (str) – FontAwesome icon code

class vstutils.api.fields.RedirectCharField(*args, **kwargs)[source]

Field for redirect by string. Often used in actions for redirect after execution.

Note

Effective only in GUI. Works similar to rest_framework.IntegerField in API.

class vstutils.api.fields.RedirectFieldMixin(**kwargs)[source]

Field mixin indicates that this field is used to send redirect address to frontend after some action.

Parameters
  • operation_name (str) – prefix for operation_id, for example if operation_id is history_get then operation_name is history

  • depend_field (str) – name of the field that we depend on, its’ value will be used for operation_id

  • concat_field_name (bool) – if True then name of the field will be added at the end of operation_id

class vstutils.api.fields.RedirectIntegerField(*args, **kwargs)[source]

Field for redirect by id. Often used in actions for redirect after execution.

Note

Effective only in GUI. Works similar to rest_framework.IntegerField in API.

class vstutils.api.fields.RelatedListField(related_name, fields, view_type='list', serializer_class=None, **kwargs)[source]

Extends class VSTCharField. With this field you can output reverse ForeignKey relation as a list of related instances.

To use it, you specify ‘related_name’ kwarg (related_manager for reverse ForeignKey) and ‘fields’ kwarg (list or tuple of fields from related model, which needs to be included).

By default VSTCharField used to serialize all field values and represent it on frontend. You can specify serializer_class and override fields as you need. For example title, description and other field properties can be set to customize frontend behaviour.

Parameters
  • related_name (str) – name of a related manager for reverse foreign key

  • fields (list[str], tuple[str]) – list of related model fields.

  • view_type (str) – determines how field are represented on frontend. Must be either ‘list’ or ‘table’.

  • fields_custom_handlers_mapping (dict) – includes custom handlers, where key: field_name, value: callable_obj that takes params: instance[dict], fields_mapping[dict], model, field_name[str]

  • serializer_class (type) – Serializer to customize types of fields, if no serializer provided VSTCharField will be used for every field in fields list

class vstutils.api.fields.SecretFileInString(*args, **kwargs)[source]

Field extends FileInStringField, but hides it’s value in admin interface.

Value must be text (not binary) and saves in model as is.

Parameters

media_types (tuple,list) – List of MIME types to select on the user’s side. Supported syntax using *. Default: ['*/*']

Note

Take effect only in GUI. In API it would behave as VSTCharField.

class vstutils.api.fields.TextareaField(*args, **kwargs)[source]

Field containing multiline string.

Note

Effective only in GUI. Works similar to VSTCharField in API.

class vstutils.api.fields.UptimeField(*args, **kwargs)[source]

Field for some uptime(time duration), in seconds, for example.

Note

Effective only in GUI. Works similar to rest_framework.IntegerField in API.

class vstutils.api.fields.VSTCharField(*args, **kwargs)[source]

CharField (extends rest_framework.fields.CharField). This field translate any json type to string for model.

class vstutils.api.fields.WYSIWYGField(*args, **kwargs)[source]

On frontend renders https://ui.toast.com/tui-editor. Saves data as markdown and escapes all html tags.

Validators

There are validation classes for fields.

class vstutils.api.validators.FileMediaTypeValidator(extensions=None, **kwargs)[source]

Base Image Validation class. Validates media types.

Parameters

extensions (typing.Union[typing.Tuple, typing.List, None]) – Tuple or List of file extensions, that should pass the validation

Raises rest_framework.exceptions.ValidationError: in case file extension are not in the list

class vstutils.api.validators.ImageBaseSizeValidator(extensions=None, **kwargs)[source]

Validates image size To use this class for validating image width/height, rewrite self.orientation to (‘height’,) or (‘width’,) or (‘height’, ‘width’)

Raises rest_framework.exceptions.ValidationError: if not(min <= (height or width) <= max)

Parameters

extensions (typing.Union[typing.Tuple, typing.List, None]) –

class vstutils.api.validators.ImageHeightValidator(extensions=None, **kwargs)[source]

Wrapper for ImageBaseSizeValidator that validates only height

Parameters
class vstutils.api.validators.ImageOpenValidator(extensions=None, **kwargs)[source]

Image validator that checks if image can be unpacked from b64 to PIL Image obj. Won’t work if Pillow isn’t installed.

Raises rest_framework.exceptions.ValidationError if PIL throws error when trying to open image

Parameters

extensions (typing.Union[typing.Tuple, typing.List, None]) –

class vstutils.api.validators.ImageResolutionValidator(extensions=None, **kwargs)[source]

Wrapper for ImageBaseSizeValidator that validates both height and width

Parameters
  • min_height – minimal height of an image being validated

  • max_height – maximal height of an image being validated

  • min_width – minimal width of an image being validated

  • max_width – maximal width of an image being validated

  • extensions (typing.Union[typing.Tuple, typing.List, None]) –

class vstutils.api.validators.ImageValidator(extensions=None, **kwargs)[source]

Base Image Validation class Validates image format Won’t work if Pillow isn’t installed Base Image Validation class. Validates media types.

Parameters

extensions (typing.Union[typing.Tuple, typing.List, None]) – Tuple or List of file extensions, that should pass the validation

Raises rest_framework.exceptions.ValidationError: in case file extension are not in the list

property has_pillow

Check if Pillow is installed

class vstutils.api.validators.ImageWidthValidator(extensions=None, **kwargs)[source]

Wrapper for ImageBaseSizeValidator that validates only width

Parameters
class vstutils.api.validators.RegularExpressionValidator(regexp=None)[source]

Class for regular expression based validation

Raises

rest_framework.exceptions.ValidationError – in case value does not match regular expression

Parameters

regexp (typing.Optional[typing.Pattern]) –

class vstutils.api.validators.UrlQueryStringValidator(regexp=None)[source]

Class for validation url query string, for example a=&b=1

Parameters

regexp (typing.Optional[typing.Pattern]) –

vstutils.api.validators.resize_image(img, width, height)[source]

Utility function to resize image proportional to specific values. Can create white margins if it’s needed to satisfy required size

Parameters
  • img (PIL.Image) – Pillow Image object

  • width (int) – Required width

  • height (int) – Required height

Returns

Pillow Image object

Return type

PIL.Image

vstutils.api.validators.resize_image_from_to(img, limits)[source]

Utility function to resize image proportional to values between min and max values for each side. Can create white margins if it’s needed to satisfy restrictions

Parameters
  • img (PIL.Image) – Pillow Image object

  • limits (dict) – Dict with min/max side restrictions like: {'width': {'min': 300, 'max: 600'}, 'height':  {'min': 400, 'max: 800'}}

Returns

Pillow Image object

Return type

PIL.Image

Serializers

Default serializer classes for web-api. Read more in Django REST Framework documentation for Serializers.

class vstutils.api.serializers.BaseSerializer(*args, **kwargs)[source]

Default serializer with logic to work with objects. Read more in DRF serializer’s documentation how to create Serializers and work with them.

class vstutils.api.serializers.EmptySerializer(*args, **kwargs)[source]

Default serializer for empty responses. In generated GUI this means that action button which will not show additional view before execution.

class vstutils.api.serializers.VSTSerializer(*args, **kwargs)[source]

Default model serializer based on rest_framework.serializers.ModelSerializer. Read more in DRF documentation how to create Model Serializers. This serializer matches model fields to extended set of serializer fields. List of available pairs specified in VSTSerializer.serializer_field_mapping. For example, to set vstutils.api.fields.FkModelField in serializer use vstutils.models.fields.FkModelField in a model.

Views

Default ViewSets for web-api.

class vstutils.api.base.CopyMixin(**kwargs)[source]

Mixin for viewsets which adds copy endpoint to view.

copy(request, **kwargs)[source]

Endpoint which copy instance with deps.

Parameters

request (rest_framework.request.Request) –

Return type

vstutils.api.responses.BaseResponseClass

copy_field_name = 'name'

Name of field which will get a prefix.

copy_prefix = 'copy-'

Value of prefix which will be added to new instance name.

List of related names which will be copied to new instance.

class vstutils.api.base.FileResponseRetrieveMixin(**kwargs)[source]

ViewSet mixin for retriving FileResponse from models with file fields data.

Example:

import datetime
import os
from django.db import models
from django_filters import ChoiceFilter
from rest_framework import permissions
from vstutils.custom_model import ListModel, FileModel
from vstutils.api import fields, base


class FileViewMixin(base.FileResponseRetrieveMixin):
    # required always
    instance_field_data = 'value'
    # required for response caching in browser
    instance_field_timestamp = 'updated'
    # this is not required, but allow to understand file response mime type in schema
    produces_for_retrieve = ['application/octet-stream', 'application/pdf']
    # search this field in instance for response filename
    # instance_field_filename = 'filename'  # as default
    # headers for response caching (default works with user auth)
    # cache_control_header_data = 'private, no-cache'
    #
    # WARNING:

class ListOfFiles(ListModel):
    data = [
        {'id': i, 'value': 'File data', 'updated': datetime.datetime(2021, 3, 1, 16, 15, 51, 801564), 'test': 1}
        for i in range(1)
    ]
    id = models.IntegerField(primary_key=True)
    value = models.TextField()
    test = models.CharField(choices=tuple((i, i) for i in range(3)), max_length=3)
    updated = models.DateTimeField()

    class Meta:
        _view_class = (FileViewMixin, 'read_only')
        _list_fields = _detail_fields = ('id', 'filename', 'test',)
        _permission_classes = [permissions.AllowAny]
        _override_permission_classes = True
        _filterset_fields = {
            'id': None,
            'test': None,
serializer_class_retrieve

alias of FileResponse

class vstutils.api.base.GenericViewSet(**kwargs)[source]

The base class for all views. Extends the standard features of the DRF class. Here are some of the possibilities:

  • Provides model attribute instead of queryset.

  • Provides to set serializers for each action separately through a dictionary action_serializers or attributes starting with serializer_class_[action name].

  • Provides to specify a serializer for lists and detail views separately.

  • Optimizes the database query for GET requests, if possible, by selecting only the fields necessary for the serializer.

create_action_serializer(*args, **kwargs)[source]

A method that implements the standard logic for actions. It relies on the passed arguments to build logic. So, if the named argument data was passed, then the serializer will be validated and saved.

Parameters
  • autosave (bool) – Enables / disables the execution of saving by the serializer if named argument data passed. Enabled by default.

  • custom_data (dict) – Dict with data which will passed to validated_data without validation.

  • serializer_class (None,type[rest_framework.serializers.Serializer]) – Serializer class for this execution. May be usefull when request and response serializers is different.

Param

data: Default serializer class argument with serializable data. Enables validation and saving.

Param

instance: Default serializer class argument with serializable instance.

Returns

Ready serializer with default logic performed.

Return type

rest_framework.serializers.Serializer

get_serializer(*args, **kwargs)[source]

Return the serializer instance that should be used for validating and deserializing input, and for serializing output.

Provide to use django.http.StreamingHttpResponse as serializer init.

Parameters
Return type

rest_framework.serializers.BaseSerializer

get_serializer_class()[source]

Provides to setup serializer class for each action.

nested_allow_check()[source]

Just raise or pass. Used for nested views for easier access checking.

class vstutils.api.base.HistoryModelViewSet(**kwargs)[source]

Default viewset like ReadOnlyModelViewSet but for historical data (allow to delete, but can’t create and update). Inherited from GenericViewSet.

class vstutils.api.base.ModelViewSet(**kwargs)[source]

A viewset that provides CRUD actions under model. Inherited from GenericViewSet.

Variables
Examples:
from vstutils.api.base import ModelViewSet
from . import serializers as sers


class StageViewSet(ModelViewSet):
    # This is difference with DRF:
    # we use model instead of queryset
    model = sers.models.Stage
    # Serializer for list view (view for a list of Model instances
    serializer_class = sers.StageSerializer
    # Serializer for page view (view for one Model instance).
    # This property is not required, if its value is the same as `serializer_class`.
    serializer_class_one = sers.StageSerializer
    # Allowed to set decorator to custom endpoint like this:
    # serializer_class_create - for create method
    # serializer_class_copy - for detail endpoint `copy`.
    # etc...
class vstutils.api.base.ReadOnlyModelViewSet(**kwargs)[source]

Default viewset like vstutils.api.base.ModelViewSet for readonly models. Inherited from GenericViewSet.

class vstutils.api.decorators.nested_view(name, arg=None, methods=None, *args, **kwargs)[source]

By default DRF does not support nested views. This decorator solves this problem.

You need two or more models with nested relationship (Many-to-Many or Many-to-One) and two viewsets. Decorator nests viewset to parent viewset class and generate paths in API.

Parameters

Note

Some view methods will not calling for performance reasons. This also applies to some of the class attributes that are usually initialized in the methods. For example, .initial() will never called. Each viewset wrapped by nested class with additional logic.

Example:

from vstutils.api.decorators import nested_view
from vstutils.api.base import ModelViewSet
from . import serializers as sers


class StageViewSet(ModelViewSet):
    model = sers.models.Stage
    serializer_class = sers.StageSerializer


nested_view('stages', 'id', view=StageViewSet)
class TaskViewSet(ModelViewSet):
    model = sers.models.Task
    serializer_class = sers.TaskSerializer

This code generates api paths:

  • /tasks/ - GET,POST

  • /tasks/{id}/ - GET,PUT,PATCH,DELETE

  • /tasks/{id}/stages/ - GET,POST

  • /tasks/{id}/stages/{stages_id}/ - GET,PUT,PATCH,DELETE

vstutils.api.decorators.subaction(*args, **kwargs)[source]

Decorator which wrap object method to subaction of viewset.

Parameters
  • methods (list) – List of allowed HTTP-request methods. Default is ["post"].

  • detail (bool) – Flag to set method execution to one instance.

  • serializer_class (vstutils.api.serializers.VSTSerializer) – Serializer for this action.

  • permission_classes – Tuple or list permission classes.

  • url_path (str) – API-path name for this action.

  • description (str) – Description for this action in OpenAPI.

  • multiaction (bool) – Allow to use this action in multiactions. Works only with vstutils.api.serializers.EmptySerializer as response.

  • require_confirmation (bool) – Sets whether the action must be confirmed before being executed.

Filtersets

For greater development convenience, the framework provides additional classes and functions for filtering elements by fields.

class vstutils.api.filters.DefaultIDFilter(data=None, queryset=None, *, request=None, prefix=None)[source]

Basic filterset to search by id. Provides a search for multiple values separated by commas. Uses extra_filter() in fields.

class vstutils.api.filters.DefaultNameFilter(data=None, queryset=None, *, request=None, prefix=None)[source]

Basic filterset to search by part of name. Uses LIKE DB condition by name_filter().

class vstutils.api.filters.FkFilterHandler(related_pk='id', related_name='name', pk_handler=<class 'int'>)[source]

Simple handler for filtering by relational fields.

Parameters
  • related_pk (str) – Field name of related model’s primary key. Default is ‘id’.

  • related_name (str) – Field name of related model’s charfield. Default is ‘name’.

  • pk_handler (typing.Callable) – Changes handler for checking value before search. Sends “0” in handler falls. Default is ‘int()’.

Example:
class CustomFilterSet(filters.FilterSet):
    author = CharFilter(method=vst_filters.FkFilterHandler(related_pk='pk', related_name='email'))

Where author is ForeignKey to User and you want to search by primary key and email.

vstutils.api.filters.extra_filter(queryset, field, value)[source]

Method for searching values in a comma-separated list.

Parameters
  • queryset (django.db.models.query.QuerySet) – model queryset for filtration.

  • field (str) – field name in FilterSet. Also supports __not suffix.

  • value (str) – comma separated list of searching values.

Returns

filtered queryset.

Return type

django.db.models.query.QuerySet

vstutils.api.filters.name_filter(queryset, field, value)[source]

Method for searching by part of name. Uses LIKE DB condition or contains qs-expression.

Parameters
  • queryset (django.db.models.query.QuerySet) – model queryset for filtration.

  • field (str) – field name in FilterSet. Also supports __not suffix.

  • value (str) – searching part of name.

Returns

filtered queryset.

Return type

django.db.models.query.QuerySet

Responses

DRF provides a standard set of variables whose names correspond to the human-readable name of the HTTP code. For convenience, we have dynamically wrapped it in a set of classes that have appropriate names and additionally provides following capabilities:

  • String responses are wrapped in json like { "detail": "string response" }.

  • Attribute timings are kept for further processing in middlewares.

  • Status code is set by class name (e.g. HTTP_200_OK or Response200 has code 200).

All classes inherit from:

class vstutils.api.responses.BaseResponseClass(*args, **kwargs)[source]

API response class with default status code.

Variables
  • status_code (int) – HTTP status code.

  • timings (int,None) – Response timings.

Parameters

timings – Response timings.

Middlewares

By default, Django supposes that a developer creates Middleware class manually, but it’s often a routine. The vstutils library offers a convenient request handler class for elegant OOP development. Middleware is used to process incoming requests and send responses before they reach final destination.

class vstutils.middleware.BaseMiddleware(get_response)[source]

Middleware base class for handling:

Middleware must be added to MIDDLEWARE list in settings.

Example:
from vstutils.middleware import BaseMiddleware
from django.http import HttpResponse


class CustomMiddleware(BaseMiddleware):
    def request_handler(self, request):
        # Add header to request
        request.headers['User-Agent'] = 'Mozilla/5.0'
        return request

    def get_response_handler(self, request):
        if not request.user.is_stuff:
            # Return 403 HTTP status for non-stuff users.
            # This request never gets in any view.
            return HttpResponse(
                "Access denied!",
                content_type="text/plain",
                status_code=403
            )
        return super().get_response_handler(request)

    def handler(self, request, response):
        # Add header to response
        response['Custom-Header'] = 'Some value'
        return response
Parameters

get_response (typing.Callable) –

get_response_handler(request)[source]

Entrypoint for breaking or continuing request handling. This function must return django.http.HttpResponse object or result of parent class calling.

Parameters

request (django.http.HttpRequest) – HTTP-request object which is wrapped from client request.

Return type

django.http.HttpResponse

handler(request, response)[source]

The response handler. Method to process responses.

Parameters
Returns

Handled response object.

Return type

django.http.HttpResponse

request_handler(request)[source]

The request handler. Called before request is handled by a view.

Parameters

request (django.http.HttpRequest) – HTTP-request object which is wrapped from client request.

Returns

Handled request object.

Return type

django.http.HttpRequest

Filter Backends

Filter Backends are used to modify model queryset. To create custom filter backend to, (i.g. annotate model queryset), you should inherit from :class:vstutils.api.filter_backends.VSTFilterBackend and override :meth:vstutils.api.filter_backends.VSTFilterBackend.filter_queryset and in some cases :meth:vstutils.api.filter_backends.VSTFilterBackend.get_schema_fields.

class vstutils.api.filter_backends.DeepViewFilterBackend[source]

Backend that filters queryset by column from deep_parent_field property of the model. Value for filtering must be provided in query param __deep_parent.

If param is missing then no filtering is applied.

If param is empty value (/?__deep_parent=) then objects with no parent (the value of the field whose name is stored in the property deep_parent_field of the model is None) returned.

This filter backend and nested view is automatically added when model has deep_parent_field property.

Example:
from django.db import models
from vstutils.models import BModel

class DeepNestedModel(BModel):
    name = models.CharField(max_length=10)
    parent = models.ForeignKey('self', null=True, default=None, on_delete=models.CASCADE)

    deep_parent_field = 'parent'
    deep_parent_allow_append = True

    class Meta:
        default_related_name = 'deepnested'

In example above if we add this model under path ‘deep’, following views will be created: /deep/ and /deep/{id}/deepnested/.

Filter backend can be used as /deep/?__deep_parent=1 and will return all DeepNestedModel objects whose parent’s primary key is 1.

You can also use generic DRF views, for that you still must set deep_parent_field to your model and manually add DeepViewFilterBackend to filter_backends list.

class vstutils.api.filter_backends.HideHiddenFilterBackend[source]

Filter Backend that hides all objects with hidden=True from the queryset

filter_queryset(request, queryset, view)[source]

Clear objects with hidden attr from queryset.

class vstutils.api.filter_backends.SelectRelatedFilterBackend[source]

Filter Backend that will automatically call prefetch_related and select_related on all relations in queryset.

filter_queryset(request, queryset, view)[source]

Select+prefetch related in queryset.

class vstutils.api.filter_backends.VSTFilterBackend[source]

A base filter backend class to be inherited from. Example:

from django.utils import timezone
from django.db.models import Value, DateTimeField

from vstutils.api.filter_backends import VSTFilterBackend

class CurrentTimeFilterBackend(VSTFilterBackend):
    def filter_queryset(self, request, queryset, view):
        return queryset.annotate(current_time=Value(timezone.now(), output_field=DateTimeField()))

In this example Filter Backend annotates time in current timezone to any connected model’s queryset.

In some cases it may be necessary to provide a parameter from a query of request. To define this parameter in the schema, you must overload the get_schema_fields function and specify a list of parameters to use.

Example:

from django.utils import timezone
from django.db.models import Value, DateTimeField

from vstutils.api.filter_backends import VSTFilterBackend

class ConstantCurrentTimeForQueryFilterBackend(VSTFilterBackend):
    query_param = 'constant'

    def filter_queryset(self, request, queryset, view):
        if self.query_param in request.query_params and request.query_params[self.query_param]:
            queryset = queryset.annotate(**{
                request.query_params[self.query_param]: Value(timezone.now(), output_field=DateTimeField())
            })
        return queryset

        def get_schema_fields(self, view):
            return [
                compat.coreapi.Field(
                    name=self.query_param,
                    required=False,
                    location='query',
                    schema=compat.coreschema.String(
                        description='Annotate value to queryset'
                    ),
                )
            ]

In this example Filter Backend annotates time in current timezone to any connected model’s queryset with field name from query constant.

get_schema_fields(view)[source]

You can also make the filter controls available to the schema autogeneration that REST framework provides, by implementing this method. The method should return a list of coreapi.Field instances.

Celery

Celery is a distributed task queue. It’s used to execute some actions asynchronously in a separate worker. For more details on Celery, check it’s official docs. For Celery related vstutils features to work, you need to specify [rpc] and [worker] sections in settings.ini. Also you need to include extra [rpc] requirements.

Tasks

class vstutils.tasks.TaskClass[source]

Wrapper for Celery BaseTask class. Usage is same as Celery standard class, but you can execute task without creating instance with TaskClass.do() method.

Example:

from vstutils.environment import get_celery_app
from vstutils.tasks import TaskClass

app = get_celery_app()

class Foo(TaskClass):
    def run(*args, **kwargs):
        return 'Foo task has been executed'

app.register_task(Foo())
Now you can call your task with various methods:
  • by executing Foo.do(*args, **kwargs)

  • get registered task instance like that - app.tasks[‘full_path.to.task.class.Foo’]

Also you can make you registered task periodic, by adding it to CELERY_BEAT_SCHEDULE in settings.py:

CELERY_BEAT_SCHEDULE = {
    'foo-execute-every-month': {
        'task': 'full_path.to.task.class.Foo',
        'schedule': crontab(day_of_month=1),
    },
}
classmethod do(*args, **kwargs)[source]

Method which send signal to celery for start remote task execution. All arguments will passed to the task TaskClass.run() method.

Return type

celery.result.AsyncResult

property name

property for proper Celery task execution, needed for TaskClass.do() method to work

run(*args, **kwargs)

The body of the task executed by workers.

Endpoint

Endpoint view has two purposes: bulk requests execution and providing openapi schema.

Endpoint url is /{API_URL}/endpoint/, for example value with default settings is /api/endpoint/.

API_URL can be changed in settings.py.

class vstutils.api.endpoint.EndpointViewSet(**kwargs)[source]

Default API-endpoint viewset.

get(request)[source]

Returns response with swagger ui or openapi json schema if ?format=openapi

Parameters

request (vstutils.api.endpoint.BulkRequestType) –

Return type

django.http.response.HttpResponse

get_client(request)[source]

Returns test client and guarantees that if bulk request comes authenticated than test client will be authenticated with the same user

Parameters

request (vstutils.api.endpoint.BulkRequestType) –

Return type

vstutils.api.endpoint.BulkClient

get_serializer(*args, **kwargs)[source]

Return the serializer instance that should be used for validating and deserializing input, and for serializing output.

Return type

vstutils.api.endpoint.OperationSerializer

get_serializer_context(context)[source]

Extra context provided to the serializer class.

Return type

dict

operate(operation_data, context)[source]

Method used to handle one operation and return result of it

Parameters
Return type

typing.Tuple[typing.Dict, typing.SupportsFloat]

post(request)[source]

Execute transactional bulk request

Parameters

request (vstutils.api.endpoint.BulkRequestType) –

Return type

vstutils.api.responses.BaseResponseClass

put(request, allow_fail=True)[source]

Execute non transaction bulk request

Parameters

request (vstutils.api.endpoint.BulkRequestType) –

Return type

vstutils.api.responses.BaseResponseClass

serializer_class

One operation serializer class.

alias of OperationSerializer

versioning_class

alias of QueryParameterVersioning

Bulk requests

Bulk request allows you send multiple reques`t to api at once, it accepts json list of operations.

Method

Transactional (all operations in one transaction)

Synchronous (operations executed one by one in given order)

PUT /{API_URL}/endpoint/

NO

YES

POST /{API_URL}/endpoint/

YES

YES

PATCH /{API_URL}/endpoint/

NO

NO

Parameters of one operation (required parameter marked by *):

  • method* - http method of request

  • path* - path of request, can be str or list

  • data - data to send

  • query - query parameters as str

  • let - string with name of variable (used for access to response result in templates)

  • headers - dict with headers which will be sent, names of headers must follow CGI specification (e.g., CONTENT_TYPE, GATEWAY_INTERFACE, HTTP_*).

  • version - str with specified version of api, if not provided then VST_API_VERSION will be used

In any request parameter you can insert result value of previous operations (<<{OPERATION_NUMBER or LET_VALUE}[path][to][value]>>), for example:

[
    {"method": "post", "path": "user", "data": {"name": "User 1"}),
    {"method": "delete", "version": "v2", "path": ["user", "<<0[data][id]>>"]}
]

Result of bulk request is json list of objects for operation:

  • method - http method

  • path - path of request, always str

  • data - data that needs to be sent

  • status - response status code

Transactional bulk request returns 502 BAG GATEWAY and does rollback after first failed request.

Warning

If you send non-transactional bulk request, you will get 200 status and must validate statuses on each operation responses.

Openapi schema

Request on GET /{API_URL}/endpoint/ returns Swagger UI.

Request on GET /{API_URL}/endpoint/?format=openapi returns openapi schema in json format. Also you can specify required version of schema using version query parameter (e.g., GET /{API_URL}/endpoint/?format=openapi&version=v2).

To change the schema after generating and before sending to user use hooks. Define one or more function, each taking 2 named arguments:

  • request - user request object.

  • schema - ordered dict with openapi schema.

Note

Sometimes hooks may raise an exception; in order to keep a chain of data modification, such exceptions are handled. The changes made to the schema before the exception however, are saved.

Example hook:
def hook_add_username_to_guiname(request, schema):
    schema['info']['title'] = f"{request.username} - {schema['info']['title']}"

To connect hook(s) to your app add function import name to the OPENAPI_HOOKS list in settings.py

OPENAPI_HOOKS = [
    '{{appName}}.openapi.hook_add_username_to_guiname',
]

Testing Framework

VST Utils Framework includes a helper in base testcase class and improves support for making API requests. That means if you want make bulk request to endpoint you dont need create and init test client, but just need to call:

endpoint_results = self.bulk([
    # list of endpoint requests
])

Creating test case

test.py module contains testcase classes based on vstutils.tests.BaseTestCase. At the moment, we officially support two styles of writing tests: classic and simple query wrappers with run check and runtime optimized bulk queries with manual value checking.

Simple example with classic tests

For example, if you have api endpoint like /api/v1/project/ and model Project you can write testcase like this:

from vstutils.tests import BaseTestCase


class ProjectTestCase(BaseTestCase):
    def setUp(self):
        super(ProjectTestCase, self).setUp()
        # init demo project
        self.initial_project = self.get_model_class('project.Test').objects.create(name="Test")

    def tearDown(self)
        super(ProjectTestCase, self).tearDown()
        # remove it after test
        self.initial_project.delete()

    def test_project_endpoint(self):
        # Test checks that api returns valid values
        self.list_test('/api/v1/project/', 1)
        self.details_test(
            ["project", self.initial_project.id],
            name=self.initial_project.name
        )
        # Try to create new projects and check list endpoint
        test_data = [
            {"name": f"TestProject{i}"}
            for i in range(2)
        ]
        id_list = self.mass_create("/api/v1/project/", test_data, 'name')
        self.list_test('/api/v1/project/', 1 + len(id_list))

This example demonstrates functionality of default test case class. Default projects are initialized for the fastest and most efficient result. We recommend to divide tests for different entities into different classes. This example demonstrate classic style of testing, but you can use bulks in your test cases.

Bulk requests in tests

Bulk query system is well suited for testing and executing valid queries. Previous example could be rewritten as follows:

from vstutils.tests import BaseTestCase


class ProjectTestCase(BaseTestCase):
    def setUp(self):
        super(ProjectTestCase, self).setUp()
        # init demo project
        self.initial_project = self.get_model_class('project.Test').objects.create(name="Test")

    def tearDown(self)
        super(ProjectTestCase, self).tearDown()
        # remove it after test
        self.initial_project.delete()

    def test_project_endpoint(self):
        test_data = [
            {"name": f"TestProject{i}"}
            for i in range(2)
        ]
        bulk_data = [
            {"method": "get", "path": ["project"]},
            {"method": "get", "path": ["project", self.initial_project.id]}
        ]
        bulk_data += [
            {"method": "post", "path": ["project"], "data": i}
            for i in test_data
        ]
        bulk_data.append(
            {"method": "get", "path": ["project"]}
        )
        results = self.bulk_transactional(bulk_data)

        self.assertEqual(results[0]['status'], 200)
        self.assertEqual(results[0]['data']['count'], 1)
        self.assertEqual(results[1]['status'], 200)
        self.assertEqual(results[1]['data']['name'], self.initial_project.name)

        for pos, result in enumerate(results[2:-1]):
            self.assertEqual(result['status'], 201)
            self.assertEqual(result['data']['name'], test_data[pos]['name'])

        self.assertEqual(results[-1]['status'], 200)
        self.assertEqual(results[-1]['data']['count'], 1 + len(test_data))

In this case, you have more code, but your tests are closer to GUI workflow, because vstutils-projects uses /api/endpoint/ for requests. Either way, bulk queries are much faster due to optimization; Testcase execution time is less comparing to non-bulk requests.

Test case API

class vstutils.tests.BaseTestCase(methodName='runTest')[source]

Main testcase class extends django.test.TestCase.

assertCheckDict(first, second, msg=None)[source]

Fail if the two fields in dicts are unequal as determined by the ‘==’ operator. Checks if first not contains or not equal field in second

Parameters
assertCount(iterable, count, msg=None)[source]

Call len() over iterable and check equals with count.

Parameters
assertRCode(resp, code=200, *additional_info)[source]

Fail if response code is not equal. Message is response body.

Parameters
bulk(data, code=200, **kwargs)[source]

Make non transactional bulk request and assert status code (default is 200)

Parameters
Return type

typing.Union[typing.List[typing.Dict[str, typing.Any]], str, bytes, bytearray, typing.Dict, typing.Sequence[typing.Union[typing.List[typing.Dict[str, typing.Any]], str, bytes, bytearray]]]

Returns

bulk response

bulk_transactional(data, code=200, **kwargs)[source]

Make transactional bulk request and assert status code (default is 200)

Parameters
Return type

typing.Union[typing.List[typing.Dict[str, typing.Any]], str, bytes, bytearray, typing.Dict, typing.Sequence[typing.Union[typing.List[typing.Dict[str, typing.Any]], str, bytes, bytearray]]]

Returns

bulk response

call_registration(data, **kwargs)[source]

Function for calling registration. Just got form data and headers.

Parameters
  • data (dict) – Registration form data.

  • kwargs – named arguments with request headers.

details_test(url, **kwargs)[source]

Test for get details of model. If you setup additional named arguments, the method check their equality with response data. Uses get_result() method.

Parameters
  • url – url to detail record. For example: /api/v1/project/1/ (where 1 is uniq id of project). You can use get_url() for building url.

  • kwargs – params that’s should be checked (key - field name, value - field value).

endpoint_call(data=None, method='get', code=200, **kwargs)[source]

Make request to endpoint and assert response status code if specified (default is 200). Uses get_result() method for execution.

Parameters
Return type

typing.Union[typing.List[typing.Dict[str, typing.Any]], str, bytes, bytearray, typing.Dict, typing.Sequence[typing.Union[typing.List[typing.Dict[str, typing.Any]], str, bytes, bytearray]]]

Returns

bulk response

endpoint_schema(**kwargs)[source]

Make request to schema. Returns dict with swagger data.

Parameters

version – API version for schema parser.

get_count(model, **kwargs)[source]

Simple wrapper over get_model_filter() which returns counter of items.

Parameters
Returns

number of instances in database.

Return type

int

get_model_class(model)[source]

Getting model class by string or return model arg.

Parameters

model (str,django.db.models.Model) – string which contains model name (if attribute model is set to the testcase class), module import, app.ModelName or django.db.models.Model.

Returns

Model class.

Return type

django.db.models.Model

get_model_filter(model, **kwargs)[source]

Simple wrapper over get_model_class() which returns filtered queryset from model.

Parameters
Return type

django.db.models.query.QuerySet

get_result(rtype, url, code=None, *args, **kwargs)[source]

Execute and test response code on request with returning parsed result of request. The method uses the following procedure:

  • Test client authorization (with user which creates in setUp()).

  • Executing a request (sending args and kwargs to request method).

  • Parsing the result (converts json string to python-object).

  • Checking the http status code with assertRCode() (if you have not specified it, the code will be selected in accordance with the request method from the standard set std_codes).

  • Logout client.

  • Return parsed result.

Parameters
  • rtype – request type (methods from Client cls): get, post etc.

  • url – requested url string or tuple for get_url(). You can use get_url() for url building or setup it as full string.

  • code (typing.Optional[int]) – expected return code from request.

  • relogin – execute force login and logout on each call. Default is True.

  • args – extra-args for Client class request method.

  • kwargs – extra-kwargs for Client class request method.

Return type

typing.Union[typing.List[typing.Dict[str, typing.Any]], str, bytes, bytearray, typing.Dict, typing.Sequence[typing.Union[typing.List[typing.Dict[str, typing.Any]], str, bytes, bytearray]]]

Returns

result of request.

get_url(*items)[source]

Function for creating url path based on VST_API_URL and VST_API_VERSION settings. Without arguments returns path to default version of api.

Return type

str

Returns

string like /api/v1/.../.../ where ... is args of function.

list_test(url, count)[source]

Test for get list of models. Checks only list count. Uses get_result() method.

Parameters
  • url – url to abstract layer. For example: /api/v1/project/. You can use get_url() for building url.

  • count – count of objects in DB.

models = None

Attribute with default project models module.

classmethod patch(*args, **kwargs)[source]

Simple unittest.mock.patch() class-method wrapper.

Return type

typing.AbstractContextManager[unittest.mock.Mock]

random_name()[source]

Simple function which returns uuid1 string.

Return type

str

std_codes: Dict[str, int] = {'delete': 204, 'get': 200, 'patch': 200, 'post': 201}

Default http status codes for different http methods. Uses in get_result()

class user_as(testcase, user)[source]

Context for execute bulk or something as user. The context manager overrides self.user in TestCase and revert this changes on exit.

Parameters

user (django.contrib.auth.models.AbstractUser) – new user object for execution.

Utils

This is tested set of development utilities. Utilities include a collection of code that will be useful in one way or another for developing the application. Vstutils uses mosts of these functions under the hood.

class vstutils.utils.BaseEnum(value)[source]

BaseEnum extends Enum class and used to create enum-like objects that can be used in django serializers or django models.

Example:

from vstutils.models import BModel

class ItemCLasses(BaseEnum):
    FIRST='FIRST'
    SECOND='SECOND'
    THIRD='THIRD'


class MyDjangoModel(BModel):
    item_class = models.CharField(max_length=1024, choices=ItemCLasses.to_choices())

    @property
    def is_second(self):
        # Function check is item has second class of instance
        return ItemCLasses.SECOND.is_equal(self.item_class)
class vstutils.utils.BaseVstObject[source]

Default mixin-class for custom objects which needed to get settings and cache.

classmethod get_django_settings(name, default=None)[source]

Get params from Django settings.

Parameters
  • name (str) – name of param

  • default (object) – default value of param

Returns

Param from Django settings or default.

class vstutils.utils.Dict[source]

Wrapper over dict which return JSON on conversion to string.

class vstutils.utils.Executor(stdout=-1, stderr=-2, **environ_variables)[source]

Command executor with realtime output write and line handling. By default and by design executor initialize string attribute output which will be modifyed by += operator with new lines by Executor.write_output() procedure. Override the method if you want change behavior.

Executor class supports periodically (0.01 sec) handling process and execute some checks by overriding Executor.working_handler() procedure method. If you want disable this behavior override the method by None value or use UnhandledExecutor.

exception CalledProcessError(returncode, cmd, output=None, stderr=None)

Raised when run() is called with check=True and the process returns a non-zero exit status.

Attributes:

cmd, returncode, stdout, stderr, output

property stdout

Alias for output attribute, to match stderr

execute(cmd, cwd)[source]

Execute commands and output this.

Parameters
  • cmd (list) – – list of cmd command and arguments

  • cwd (str) – – workdir for executions

Returns

– string with full output

Return type

str

working_handler(proc)[source]

Additional handler for executions.

Parameters

proc (subprocess.Popen) –

Return type

None

write_output(line)[source]
Parameters

line (str) – – line from command output

Returns

None

Return type

None

class vstutils.utils.KVExchanger(key, timeout=None)[source]

Class for transmit data using key-value fast (cache-like) storage between services. Uses same cache-backend as Lock.

class vstutils.utils.Lock(id, payload=1, repeat=1, err_msg='', timeout=None)[source]

Lock class for multi-jobs workflow. Based on KVExchanger. The Lock allows only one thread to enter the part that’s locked and shared between apps using one locks cache (see also [locks]).

Parameters
  • id (int,str) – – unique id for lock.

  • payload – – lock additional info. Should be any boolean True value.

  • repeat (int) – – time to wait lock.release. Default 1 sec.

  • err_msg (str) – – message for AcquireLockException error.

Note

  • Used django.core.cache lib and settings in settings.py

  • Have Lock.SCHEDULER and Lock.GLOBAL id

Example:
from vstutils.utils import Lock

with Lock("some_lock_identifier", repeat=30, err_msg="Locked by another proccess") as lock:
    # where
    # ``"some_lock_identifier"`` is unique id for lock and
    # ``30`` seconds lock is going wait until another process will release lock id.
    # After 30 sec waiting lock will raised with :class:`.Lock.AcquireLockException`
    # and ``err_msg`` value as text.
    some_code_execution()
    # ``lock`` object will has been automatically released after
    # exiting from context.
Another example without context manager:
from vstutils.utils import Lock

# locked block after locked object created
lock = Lock("some_lock_identifier", repeat=30, err_msg="Locked by another proccess")
# deleting of object calls ``lock.release()`` which release and remove lock from id.
del lock
exception AcquireLockException[source]

Exception which will raised on unreleased lock.

class vstutils.utils.ModelHandlers(type_name, err_message=None)[source]

Handlers for some models like ‘INTEGRATIONS’ or ‘REPO_BACKENDS’. Based on ObjectHandlers but more specific for working with models. All handlers backends get by first argument model object.

Attributes:

Parameters
  • objects (dict) – – dict of objects like: {<name>: <backend_class>}

  • keys (list) – – names of supported backends

  • values (list) – – supported backends classes

  • type_name (str) –

  • err_message (typing.Optional[str]) –

get_object(name, obj)[source]
Parameters
Returns

backend object

Return type

object

class vstutils.utils.ObjectHandlers(type_name, err_message=None)[source]

Handlers wrapper for get objects from some settings structure.

Example:
from vstutils.utils import ObjectHandlers

'''
In `settings.py` you should write some structure:

SOME_HANDLERS = {
    "one": {
        "BACKEND": "full.python.path.to.module.SomeClass"
    },
    "two": {
        "BACKEND": "full.python.path.to.module.SomeAnotherClass",
        "OPTIONS": {
            "some_named_arg": "value"
        }
    }
}
'''

handlers = ObjectHandlers('SOME_HANDLERS')

# Get class handler for 'one'
one_backend_class = handlers['one']
# Get object of backend 'two'
two_obj = handlers.get_object()
# Get object of backend 'two' with overriding constructor named arg
two_obj_overrided = handlers.get_object(some_named_arg='another_value')
Parameters
backend(name)[source]

Get backend class

Parameters

name (str) – – name of backend type

Returns

class of backend

Return type

type,types.ModuleType,object

class vstutils.utils.Paginator(qs, chunk_size=None)[source]

Class for fragmenting the query for small queries.

class vstutils.utils.SecurePickling(secure_key=None)[source]

Secured pickle wrapper by Vigenère cipher.

Warning

Do not use it with untrusted transport anyway.

Example:
from vstutils.utils import SecurePickling


serializer = SecurePickling('password')

# Init secret object
a = {"key": "value"}
# Serialize object with secret key
pickled = serializer.dumps(a)
# Deserialize object
unpickled = serializer.loads(pickled)

# Check, that object is correct
assert a == unpickled
Parameters

secure_key (typing.Optional[str]) –

class vstutils.utils.URLHandlers(type_name='URLS', *args, **kwargs)[source]

Object handler for GUI views. Uses GUI_VIEWS from settings.py. Based on ObjectHandlers but more specific to urlpatterns.

Example:
from vstutils.utils import URLHandlers


# By default gets from `GUI_VIEWS` in `settings.py`
urlpatterns = list(URLHandlers())
Parameters

type_name (str) –

get_object(name, *argv, **kwargs)[source]

Get url object tuple for urls.py

Parameters
  • name (str) – url regexp from

  • argv – overrided args

  • kwargs – overrided kwargs

Returns

url object

Return type

django.urls.re_path

class vstutils.utils.UnhandledExecutor(stdout=-1, stderr=-2, **environ_variables)[source]

Class based on Executor but disables working_handler.

class vstutils.utils.apply_decorators(*decorators)[source]

Decorator which apply list of decorators on method or class.

Example:
from vstutils.utils import apply_decorators

def decorator_one(func):
    print(f"Decorated {func.__name__} by first decorator.")
    return func

def decorator_two(func):
    print(f"Decorated {func.__name__} by second decorator.")
    return func

@apply_decorators(decorator_one, decorator_two)
def decorated_function():
    # Function decorated by both decorators.
    print("Function call.")
class vstutils.utils.classproperty(fget, fset=None)[source]

Decorator which makes class method as class property.

Example:
from vstutils.utils import classproperty

class SomeClass(metaclass=classproperty.meta):
    # Metaclass is needed for set attrs in class
    # instead of and not only object.

    some_value = None

    @classproperty
    def value(cls):
        return cls.some_value

    @value.setter
    def value(cls, new_value):
        cls.some_value = new_value
Parameters
vstutils.utils.create_view(model, **meta_options)[source]

A simple function for getting the generated view by standard means, but with overloaded meta-parameters. This method can completely get rid of the creation of proxy models.

Example:
from vstutils.utils import create_view

from .models import Host

# Host model has full :class:`vstutils.api.base.ModelViewSet` view.
# For overriding and create simple list view just setup this:
HostListViewSet = create_view(
    HostList,
    view_class='list_only'
)

Note

This method is also recommended in cases where there is a problem of recursive imports.

Parameters

model (Type[vstutils.models.BaseModel]) – Model class with .get_view_class method. This method also has vstutils.models.BModel.

Return type

vstutils.api.base.GenericViewSet

vstutils.utils.decode(key, enc)[source]

Decode string from encoded by Vigenère cipher.

Parameters
  • key (str) – – secret key for encoding

  • enc (str) – – encoded string for decoding

Returns

– decoded string

Return type

str

vstutils.utils.deprecated(func)[source]

This is a decorator which can be used to mark functions as deprecated. It will result in a warning being emitted when the function is used.

Parameters

func (typing.Callable) –

vstutils.utils.encode(key, clear)[source]

Encode string by Vigenère cipher.

Parameters
  • key (str) – – secret key for encoding

  • clear (str) – – clear value for encoding

Returns

– encoded string

Return type

str

vstutils.utils.get_render(name, data, trans='en')[source]

Render string from template.

Parameters
  • name (str) – – full template name

  • data (dict) – – dict of rendered vars

  • trans (str) – – translation for render. Default ‘en’.

Returns

– rendered string

Return type

str

vstutils.utils.lazy_translate(text)[source]

The lazy_translate function has the same behavior as translate(), but wraps it in a lazy promise.

This is very useful, for example, for translating error messages in class attributes before the language code is known.

Parameters

text (str) – Text message which should be translated.

Return type

django.utils.functional.Promise

vstutils.utils.list_to_choices(items_list, response_type=<class 'list'>)[source]

Method to create django model choices from flat list of values.

Parameters
Return type

typing.Iterable[typing.Tuple[str, str]]

Returns

list of tuples from items_list values

class vstutils.utils.model_lock_decorator(**kwargs)[source]

Decorator for functions where ‘pk’ kwarg exist for lock by id.

Warning

  • On locked error raised Lock.AcquireLockException

  • Method must have and called with pk named arg.

class vstutils.utils.raise_context(*args, **kwargs)[source]

Context for exclude exceptions.

class vstutils.utils.raise_context_decorator_with_default(*args, **kwargs)[source]

Context for exclude errors and return default value.

Example:
from yaml import load
from vstutils.utils import raise_context_decorator_with_default


@raise_context_decorator_with_default(default={})
def get_host_data(yaml_path, host):
    with open(yaml_path, 'r') as fd:
        data = load(fd.read(), Loader=Loader)
    return data[host]
    # This decorator used when you must return some value even on error
    # In log you also can see traceback for error if it occur

def clone_host_data(host):
    bs_data = get_host_data('inventories/aws/hosts.yml', 'build_server')
    ...
class vstutils.utils.redirect_stdany(new_stream=<_io.StringIO object>, streams=None)[source]

Context for redirect any output to own stream.

Note

  • On context return stream object.

  • On exit return old streams

vstutils.utils.send_mail(subject, message, from_email, recipient_list, fail_silently=False, auth_user=None, auth_password=None, connection=None, html_message=None, **kwargs)[source]

Wrapper over django.core.mail.send_mail() which provide additional named arguments.

vstutils.utils.send_template_email(sync=False, **kwargs)[source]

Function executing sync or async email sending; according sync argument and settings variable “RPC_ENABLED”. You can use this function to send message, it sends message asynchronously or synchronously. If you don’t set settings for celery or don’t have celery it sends synchronously mail. If celery is installed and configured and sync argument of the function is set to False, it sends asynchronously email.

Parameters
  • sync (bool) – argument for determining how send email, asynchronously or synchronously

  • subject – mail subject.

  • email – list of strings or single string, with email addresses of recipients

  • template_name – relative path to template in templates directory, must include extension in file name.

  • context_data – dictionary with context for rendering message template.

vstutils.utils.send_template_email_handler(subject, email_from, email, template_name, context_data=None, **kwargs)[source]

Function for email sending. The function convert recipient to list and set context before sending if it possible.

Parameters
  • subject (str) – mail subject.

  • email_from (str) – sender that be setup in email.

  • email (typing.Union[typing.List, str]) – list of strings or single string, with email addresses of recipients

  • template_name (str) – relative path to template in templates directory, must include extension in file name.

  • context_data (typing.Optional[typing.Dict]) – dictionary with context for rendering message template.

  • kwargs – additional named arguments for send_mail

Return type

typing.SupportsInt

Returns

Number of emails sent.

class vstutils.utils.tmp_file(data='', mode='w', bufsize=-1, **kwargs)[source]

Temporary file with name generated and auto removed on close.

Attributes:

Parameters
  • data (str) – – string to write in tmp file.

  • mode (str) – – file open mode. Default ‘w’.

  • bufsize (int) – – bufer size for tempfile.NamedTemporaryFile

  • kwargs – – other kwargs for tempfile.NamedTemporaryFile

write(wr_string)[source]

Write to file and flush

Parameters

wr_string (str) – – writable string

Returns

None

Return type

None

class vstutils.utils.tmp_file_context(*args, **kwargs)[source]

Context object for work with tmp_file. Auto close on exit from context and remove if file still exist.

This context manager over tmp_file

vstutils.utils.translate(text)[source]

The translate function supports translation message dynamically with standard i18n vstutils’es mechanisms usage.

Uses django.utils.translation.get_language() to get the language code and tries to get the translation from the list of available ones.

Parameters

text (str) – Text message which should be translated.

Return type

str