Backend API manual¶

VST Utils framework uses Django, Django Rest Framework, drf-yasg and Celery.

Models¶

A model is the single, definitive source of truth about your data. It contains essential fields and behavior for the data you’re storing. Usually best practice is to avoid writing views and serializers manually, as BModel provides plenty of Meta attributes to autogenerate serializers and views for many use cases.

Default Django model classes overrides in vstutils.models module.

class vstutils.models.BModel(*args, **kwargs)[source]¶

Default model class that generates model viewset, separate serializers for list() and retrieve(), filters, api endpoints and nested views.

Examples:

from django.db import models
from rest_framework.fields import ChoiceField
from vstutils.models import BModel

class Stage(BModel):
    name = models.CharField(max_length=256)
    order = models.IntegerField(default=0)

    class Meta:
        default_related_name = "stage"
        ordering = ('order', 'id',)
        # fields which would be showed on list.
        _list_fields = [
            'id',
            'name',
        ]
        # fields which would be showed on detail view and creation.
        _detail_fields = [
            'id',
            'name',
            'order'
        ]
        # make order as choices from 0 to 9
        _override_detail_fields = {
            'order': ChoiceField((str(i) for i in range(10)))
        }


class Task(BModel):
    name = models.CharField(max_length=256)
    stages = models.ManyToManyField(Stage)
    _translate_model = 'Task'

    class Meta:
        # fields which would be showed.
        _list_fields = [
            'id',
            'name',
        ]
        # create nested views from models
        _nested = {
            'stage': {
                'allow_append': False,
                'model': Stage
            }
        }

In this case, you create models which could converted to simple view, where:

POST/GET to /api/version/task/ - creates new or get list of tasks
PUT/PATCH/GET/DELETE to /api/version/task/:id/ - updates, retrieves or removes instance of task
POST/GET to /api/version/task/:id/stage/ - creates new or get list of stages in task
PUT/PATCH/GET/DELETE to /api/version/task/:id/stage/:stage_id - updates, retrieves or removes instance of stage in task.

To attach a view to an API insert the following code in settings.py:

API[VST_API_VERSION][r'task'] = {
    'model': 'your_application.models.Task'
}

For primary access to generated view inherit from Task.generated_view property.

To make translation on frontend easier use _translate_model attribute with model_name

List of meta-attributes for generating a view:

_view_class - list of additional view classes to inherit from, class or string to import with base class ViewSet. Constants are also supported:
- read_only - to create a view only for viewing;
- list_only - to create a view with list only;
- history - to create a view only for viewing and deleting records.
CRUD-view is applied by default.
_serializer_class - class of API serializer; use this attribute to specify parent class for autogenerated serializers. Default is vstutils.api.serializers.VSTSerializer. Can take a string to import, serializer class or django.utils.functional.SimpleLazyObject.
_serializer_class_name - model name for OpenAPI definitions.This would be a model name in generated admin interface. Default is name of model class.
_list_fields or _detail_fields - list of fields which will be listed in entity list or detail view accordingly. Same as DRF serializers meta-attribute “fields”.
_override_list_fields or _override_detail_fields - mapping with names and field types that will be redeclared in serializer attributes(think of it as declaring fields in DRF ModelSerializer).
_properties_groups - dict with key as group name and value as list of fields(str). Allows to group fields in sections on frontend.
_view_field_name - name of field frontend shows as main view name.
_non_bulk_methods - list of methods which must not used via bulk requests.
_extra_serializer_classes - mapping with additional serializers in viewset. For example, custom serializer, which will compute smth in action (mapping name). Value can be string for import. Important note: setting model attribute to None allows to use standard serializer generation mechanism and get fields from a list or detail serializer (set __inject_from__ serializer’s meta attribute to list or detail accordingly).
_filterset_fields - list/dict of filterset names for API-filtering. Default is list of fields in list view. During processing a list of fields checks for the presence of special field names and inherit additional parent classes. If the list contains id, class will inherit from vstutils.api.filters.DefaultIDFilter. If the list contains name, class will inherit from vstutils.api.filters.DefaultNameFilter. If both conditions are present, inheritance will be from all of the above classes. Possible values include list of fields to filter or dict where key is a field name and value is a Filter class. Dict extends attribute functionality and provides ability to override filter field class (None value disables overriding).
_search_fields - tuple or list of fields using for search requests. By default (or None) get all filterable fields in detail view.
_filter_backends - list of vstutils.api.filter_backends.VSTFilterBackend classes. Values can be string for import.
_override_filter_backends - boolean flag indicates that _filter_backends override default viewset filter_backends (otherwise appends). Default is False.
_permission_classes - list of DRF permission classes. Values can be string for import.
_override_permission_classes - boolean flag indicates that _permission_classes override default viewset (otherwise appends). Default is False.
_copy_attrs - list of model-instance attributes indicates that object is copiable with this attrs.
_nested - key-value mapping with nested views (key - nested name, kwargs for vstutils.api.decorators.nested_view decorator but supports model attribute as nested). model can be string for import.

Note

You may need to create an action on generated view. Use vstutils.models.decorators.register_view_action decorator with the detail argument to determine applicability to a list or detail entry. In this case, the decorated method will take an instance of the view object as self attribute.

Note

In some cases, inheriting models may require to inherit Meta class from the base model. If the Meta is explicitly declared in the base class, then you can get it through the attribute OriginalMeta and use it for inheritance.

Note

Docstring of model will be reused for view descriptions. It is possible to write both a general description for all actions and description for each action using the following syntax:

General description for all actions.

action_name:
    Description for this action.

another_action:
    Description for another action.

hidden¶: If hidden is set to True, entry will be excluded from query in BQuerySet.

id¶: Primary field for select and search in API.

class vstutils.models.Manager(*args, **kwargs)[source]¶: Default VSTUtils manager. Used by BaseModel and BModel. Uses BQuerySet as base.

class vstutils.models.queryset.BQuerySet(model=None, query=None, using=None, hints=None)[source]¶

Represent a lazy database lookup for a set of objects. Allows to override default iterable class by custom_iterable_class attribute (class with __iter__ method which returns generator of model objects) and default query class by custom_query_class attribute (class inherited from django.db.models.sql.query.Query).

cleared()[source]¶: Filter queryset for models with attribute ‘hidden’ and exclude all hidden objects.

get_paginator(*args, **kwargs)[source]¶: Returns initialized object of vstutils.utils.Paginator over current instance’s QuerySet. All args and kwargs go to to Paginator’s constructor.

paged(*args, **kwargs)[source]¶: Returns paginated data with custom Paginator-class. Uses PAGE_LIMIT from global settings by default.

class vstutils.models.decorators.register_view_action(*args, **kwargs)[source]¶: Decorator for turning model methods to generated view actions. The decorated method becomes a method of generated view and self is an view object. See supported args in vstutils.api.decorators.subaction()

Note

Often, an action does not transfer any parameters and requires only sending an empty query. To speed up development, we set the default serializer to vstutils.api.serializers.EmptySerializer.

You can also use custom models without using database:¶

class vstutils.custom_model.FileModel(*args, **kwargs)[source]¶

Custom model which loads data from YAML-file instead of database. Path to the file stored in FileModel.file_path attribute.

Examples:

Source file stored in /etc/authors.yaml with content:

- name: "Sergey Klyuykov"
- name: "Michael Taran"

Example:

from vstutils.custom_model import FileModel, CharField

class Authors(FileModel):
    name = CharField(max_length=512)

    file_path = '/etc/authors.yaml'

class vstutils.custom_model.ListModel(*args, **kwargs)[source]¶

Custom model which uses a list of dicts with data (attribute ListModel.data) instead of database records. Useful when you have a simple list of data.

Examples:

from vstutils.custom_model import ListModel, CharField


class Authors(ListModel):
    name = CharField(max_length=512)

    data = [
        {"name": "Sergey Klyuykov"},
        {"name": "Michael Taran"},
    ]

data = []¶: List with data dicts. Empty by default.

Model Fields¶

class vstutils.models.fields.FkModelField(to, on_delete, related_name=None, related_query_name=None, limit_choices_to=None, parent_link=False, to_field=None, db_constraint=True, **kwargs)[source]¶: Extends django.db.models.ForeignKey. Use this field in vstutils.models.BModel to get vstutils.api.FkModelField in serializer. To set Foreign Key relation set to argument to string path to model or to Model Class as in django.db.models.ForeignKey

class vstutils.models.fields.MultipleFieldFile(instance, field, name)[source]¶

Subclasses django.db.models.fields.files.FieldFile. Provides MultipleFieldFile.save() and MultipleFieldFile.delete() to manipulate the underlying file, as well as update the associated model instance.

delete(save=True)[source]¶: Delete file from storage and from object attr.

save(name, content, save=True)[source]¶: Save changes in file to storage and to object attr.

class vstutils.models.fields.MultipleFileDescriptor(field)[source]¶

Subclasses django.db.models.fields.files.FileDescriptor to handle list of files. Return a list of MultipleFieldFile when accessed so you can write code like:

from myapp.models import MyModel
instance = MyModel.objects.get(pk=1)
instance.files[0].size

get_file(file, instance)[source]¶: Always return valid attr_class object.For details on logic see django.db.models.fields.files.FileDescriptor.__get__().

class vstutils.models.fields.MultipleFileField(**kwargs)[source]¶

Subclasses django.db.models.fields.files.FileField. Field for storing a list of Storage-kept files. All args passed to FileField.

attr_class¶: alias of vstutils.models.fields.MultipleFieldFile

descriptor_class¶: alias of vstutils.models.fields.MultipleFileDescriptor

class vstutils.models.fields.MultipleFileMixin(**kwargs)[source]¶

Mixin suited to use with django.db.models.fields.files.FieldFile to transform it to a Field with list of files.

get_prep_value(value)[source]¶: Prepare value for database insertion

pre_save(model_instance, add)[source]¶: Call .save() method on every file in list

class vstutils.models.fields.MultipleImageField(**kwargs)[source]¶

Field for storing a list of storage-kept images. All args are passed to django.db.models.fields.files.ImageField, except height_field and width_field, they are not currently implemented.

attr_class¶: alias of vstutils.models.fields.MultipleImageFieldFile

descriptor_class¶: alias of vstutils.models.fields.MultipleFileDescriptor

class vstutils.models.fields.MultipleImageFieldFile(instance, field, name)[source]¶: Subclasses MultipleFieldFile and ImageFile mixin, handles deleting _dimensions_cache when file is deleted.

class vstutils.models.fields.MultipleNamedBinaryFileInJSONField(*args, db_collation=None, **kwargs)[source]¶: Extends django.db.models.TextField. Use this field in vstutils.models.BModel to get vstutils.api.MultipleNamedBinaryFileInJSONField in serializer.

class vstutils.models.fields.MultipleNamedBinaryImageInJSONField(*args, db_collation=None, **kwargs)[source]¶: Extends django.db.models.TextField. Use this field in vstutils.models.BModel to get vstutils.api.MultipleNamedBinaryImageInJSONField in serializer.

class vstutils.models.fields.NamedBinaryFileInJSONField(*args, db_collation=None, **kwargs)[source]¶: Extends django.db.models.TextField. Use this field in vstutils.models.BModel to get vstutils.api.NamedBinaryFileInJSONField in serializer.

class vstutils.models.fields.NamedBinaryImageInJSONField(*args, db_collation=None, **kwargs)[source]¶: Extends django.db.models.TextField. Use this field in vstutils.models.BModel to get vstutils.api.NamedBinaryImageInJSONField in serializer.

Web API¶

Web API is based on Django Rest Framework with additional nested functions.

Fields¶

The Framework includes a list of convenient serializer fields. Some of them take effect only in generated admin interface.

Additional serializer fields for generating OpenAPI and GUI.

class vstutils.api.fields.AutoCompletionField(*args, **kwargs)[source]¶

Field that provides autocompletion on frontend, using specified list of objects.

Parameters

autocomplete (list,tuple,str) – Autocompletion reference. You can set list/tuple with values or set OpenApi schema definition name. For definition name GUI will find optimal link and will show values based on autocomplete_property and autocomplete_represent arguments.
autocomplete_property (str) – this argument indicates which attribute will be get from OpenApi schema definition model as value.
autocomplete_represent – this argument indicates which attribute will be get from OpenApi schema definition model as represent value.
use_prefetch (bool) – prefetch values on frontend at list-view. Default is True.

Note

Effective only in GUI. Works similar to VSTCharField in API.

class vstutils.api.fields.BinFileInStringField(*args, **kwargs)[source]¶: Field extends FileInStringField, but work with binary(base64) files.

Note

Effective only in GUI. Works similar to VSTCharField in API.

class vstutils.api.fields.CommaMultiSelect(*args, **kwargs)[source]¶

Field containing a list of values with specified separator(default: “,”). Gets list of values from another model or custom list. Provides autocompletion as AutoCompletionField, but with comma-lists. Suited for property-fields in model where main logic is already implemented or with CharField.

Parameters

select (str,tuple,list) – OpenApi schema definition name or list with values.
select_separator (str) – separator of values. Default is comma.
select_property,select_represent – work as autocomplete_property and autocomplete_represent. Default is name.
use_prefetch – prefetch values on frontend at list-view. Default is False.
make_link – Show value as link to model. Default is True.
dependence (dict) – Dictionary, where keys are name of field from the same model, and values are name of query filter .If at least one of the fields that we depend on is non nullable, required and set to null, autocompletion list will be empty and field will be disabled.

Note

Effective only in GUI. Works similar to VSTCharField in API.

class vstutils.api.fields.DependEnumField(*args, **kwargs)[source]¶

Field extends DynamicJsonTypeField but its value is not transformed to json and would be given as is. Useful for property in models or for actions.

Parameters

field (str) – field in model which value change will change type of current value.
types – key-value mapping where key is value of subscribed field and value is type (in OpenApi format) of current field.
choices (dict) – variants of choices for different subscribed field values. Uses mapping where key is value of subscribed field and value is list with values to choice.

Note

Effective only in GUI. In API works similar to VSTCharField without value modification.

class vstutils.api.fields.DependFromFkField(*args, **kwargs)[source]¶

Field extends DynamicJsonTypeField. Validates field data by field_attribute chosen in related model. By default, any value of field_attribute validates as VSTCharField. To override this behavior set dict attribute {field_attribute value}_fields_mapping in related model where:

key - string representation of value type which is received from related instance field_attribute.
value - rest_framework.Field instance for validation.

Parameters

field (str) – field in model which value change changes type of current value. Field must be FkModelField.
field_attribute (str) – attribute of model related model instance with name of type.

Warning

field_attribute in related model must be rest_framework.ChoicesField or GUI will show field as simple text.

class vstutils.api.fields.DynamicJsonTypeField(*args, **kwargs)[source]¶

Field which type is based on another field. It converts value to internal string and represent field as json object.

Parameters

field (str) – field in model which value change will change type of current value.
types – key-value mapping where key is value of subscribed field and value is type (in OpenApi format) of current field.
choices (dict) – variants of choices for different subscribed field values. Uses mapping where key is value of subscribed field and value is list with values to choice.

Note

Effective only in GUI. In API works similar to VSTCharField without value modifications.

class vstutils.api.fields.FileInStringField(*args, **kwargs)[source]¶

Field extends VSTCharField and saves file’s content as string.

Value must be text (not binary) and saves in model as is.

Note

Take effect only in GUI. In API it would behave as VSTCharField.

class vstutils.api.fields.FkField(*args, **kwargs)[source]¶

Implementation of ForeignKeyField.You can specify which field of a related model will be stored in field(default: “id”), and which will represent field on frontend.

Parameters

select (str) – OpenApi schema definition name.
autocomplete_property (str) – this argument indicates which attribute will be get from OpenApi schema definition model as value. Default is id.
autocomplete_represent – this argument indicates which attribute will be get from OpenApi schema definition model as represent value. Default is name.
use_prefetch (bool) – prefetch values on frontend at list-view. Default is True.
make_link (bool) – show value as link to model. Default is True.
dependence (dict) –
dictionary, where keys are names of a field from the same model, and keys are name of query filter. If at least one of the fields that we depend on is non nullable, required and set to null, autocompletion list will be empty and field will be disabled.

There are some special keys for dependence dictionary to get data that is stored on frontend without additional database query:

'<<pk>>', - get primary key of current instance

'<<view_name>>' get view name from Vue component,

'<<parent_view_name>>' get view name from Vue component,

'<<view_level>>' get view level,

'<<operation_id>>' get operation_id,

'<<parent_operation_id'>> get parent_operation_id

Examples:

field = FkField(select=Category, dependence={'<<pk>>': 'my_filter'})

This filter will get pk of current object and make query on frontend ‘/category?my_filter=3’ where ‘3’ is primary key of current instance.

Parameters: filters (dict) – dictionary, where keys are names of a field from a related (by this FkField) model, and values are values of that field.

Note

Intersection of dependence.values() and filters.keys() will throw error to prevent ambiguous filtering.

Note

Effective only in GUI. Works similar to rest_framework.IntegerField in API.

class vstutils.api.fields.FkModelField(*args, **kwargs)[source]¶

Extends FkField, but stores referred model class. This field is useful for django.db.models.ForeignKey fields in model to set.

Parameters

select (vstutils.models.BModel,vstutils.api.serializers.VSTSerializer) – model class (based on vstutils.models.BModel) or serializer class which used in API and has path in OpenApi schema.
autocomplete_property (str) – this argument indicates which attribute will be get from OpenApi schema definition model as value. Default is id.
autocomplete_represent – this argument indicates which attribute will be get from OpenApi schema definition model as represent value. Default is name.
use_prefetch – prefetch values on frontend at list-view. Default is True.
make_link – Show value as link to model. Default is True.

Warning

Model class get object from database during .to_internal_value execution. Be careful on mass save executions.

Warning

Permissions to model which is referred by this field, are not to be checked. You should check it manually in signals or validators.

class vstutils.api.fields.HtmlField(*args, **kwargs)[source]¶: Field contains html text and marked as format:html. The field does not validate whether its content is HTML.

Warning

To avoid vulnerability, do not allow users to modify this data because users ate able to execute their scripts.

Note

Effective only in GUI. Works similar to VSTCharField in API.

class vstutils.api.fields.MaskedField(*args, **kwargs)[source]¶

Extends class ‘rest_framework.serializers.CharField’. Field that apply mask to value

Parameters: mask (dict, str) – IMask

Note

Effective only on frontend.

class vstutils.api.fields.MultipleNamedBinaryFileInJsonField(*args, **kwargs)[source]¶

Extends NamedBinaryFileInJsonField but uses list of JSONs. Allows to operate with multiple files as list of NamedBinaryFileInJsonField.

Attrs: NamedBinaryInJsonField.file: if True, accept only subclasses of File as input. If False, accept only string input. Default: False.

file_field¶: alias of vstutils.models.fields.MultipleFieldFile

class vstutils.api.fields.MultipleNamedBinaryImageInJsonField(*args, **kwargs)[source]¶: Extends MultipleNamedBinaryFileInJsonField but uses list of JSONs. Used for operating with multiple images and works as list of NamedBinaryImageInJsonField.

class vstutils.api.fields.NamedBinaryFileInJsonField(*args, **kwargs)[source]¶

Field that takes JSON with properties: * name - string - name of file; * mediaType - string - MIME type of file * content - base64 string - content of file.

This field is useful for saving binary files with their names in django.db.models.CharField or django.db.models.TextField model fields. All manipulations with decoding and encoding binary content data executes on client. This imposes reasonable limits on file size.

Additionally, this field can construct django.core.files.uploadedfile.SimpleUploadedFile from incoming JSON and store it as file in django.db.models.FileField if file argument is set to True

Attrs: NamedBinaryInJsonField.file: if True, accept only subclasses of File as input. If False, accept only string input. Default: False.

Note

Effective only in GUI. Works similar to VSTCharField in API.

class vstutils.api.fields.NamedBinaryImageInJsonField(*args, **kwargs)[source]¶: Extends NamedBinaryFileInJsonField to represent image on frontend (if binary image is valid).Validate this field with vstutils.api.validators.ImageValidator.

class vstutils.api.fields.PasswordField(*args, **kwargs)[source]¶: Extends CharField but in schema set format to password. Show all characters as asterisks instead of real value in GUI.

class vstutils.api.fields.PhoneField(*args, **kwargs)[source]¶: Extends class ‘rest_framework.serializers.CharField’. Field for for phone in international format

class vstutils.api.fields.RatingField(min_value=0, max_value=5, step=1, front_style='stars', **kwargs)[source]¶

Extends class ‘rest_framework.serializers.FloatField’. This field represents a rating form input on frontend. Grading limits can be specified with ‘min_value=’ and ‘max_value=’, defaults are 0 to 5.Minimal step between grades are specified in ‘step=’, default - 1.Frontend visual representation can be chosen with ‘front_style=’, available variants are listed in ‘self.valid_front_styles’.

for ‘slider’ front style, you can specify slider color, by passing valid color to ‘color=’. for ‘fa_icon’ front style, you can specify FontAwesome icon that would be used for displaying rating, by passing a valid FontAwesome icon code to ‘fa_class=’.

Parameters

min_value (float, int) – minimal level
max_value (float, int) – maximal level
step (float, int) – minimal step between levels
front_style (str) – visualization on frontend field. Allowed: [‘stars’, ‘slider’, ‘fa_icon’].
color (str) – color of rating element (star, icon or slider) in css format
fa_class (str) – FontAwesome icon code

class vstutils.api.fields.RedirectCharField(*args, **kwargs)[source]¶: Field for redirect by string. Often used in actions for redirect after execution.

Note

Effective only in GUI. Works similar to rest_framework.IntegerField in API.

class vstutils.api.fields.RedirectFieldMixin(**kwargs)[source]¶

Field mixin indicates that this field is used to send redirect address to frontend after some action.

Parameters

operation_name (str) – prefix for operation_id, for example if operation_id is history_get then operation_name is history
depend_field (str) – name of the field that we depend on, its’ value will be used for operation_id
concat_field_name (bool) – if True then name of the field will be added at the end of operation_id

class vstutils.api.fields.RedirectIntegerField(*args, **kwargs)[source]¶: Field for redirect by id. Often used in actions for redirect after execution.

Note

Effective only in GUI. Works similar to rest_framework.IntegerField in API.

class vstutils.api.fields.RelatedListField(related_name, fields, view_type='list', serializer_class=None, **kwargs)[source]¶

Extends class VSTCharField. With this field you can output reverse ForeignKey relation as a list of related instances.

To use it, you specify ‘related_name’ kwarg (related_manager for reverse ForeignKey) and ‘fields’ kwarg (list or tuple of fields from related model, which needs to be included).

By default VSTCharField used to serialize all field values and represent it on frontend. You can specify serializer_class and override fields as you need. For example title, description and other field properties can be set to customize frontend behaviour.

Parameters

related_name (str) – name of a related manager for reverse foreign key
fields (list[str], tuple[str]) – list of related model fields.
view_type (str) – determines how field are represented on frontend. Must be either ‘list’ or ‘table’.
fields_custom_handlers_mapping (dict) – includes custom handlers, where key: field_name, value: callable_obj that takes params: instance[dict], fields_mapping[dict], model, field_name[str]
serializer_class (type) – Serializer to customize types of fields, if no serializer provided VSTCharField will be used for every field in fields list

class vstutils.api.fields.SecretFileInString(*args, **kwargs)[source]¶

Field extends FileInStringField, but hides it’s value in admin interface.

Value must be text (not binary) and saves in model as is.

Note

Take effect only in GUI. In API it would behave as VSTCharField.

class vstutils.api.fields.TextareaField(*args, **kwargs)[source]¶: Field containing multiline string.

Note

Effective only in GUI. Works similar to VSTCharField in API.

class vstutils.api.fields.UptimeField(*args, **kwargs)[source]¶: Field for some uptime(time duration), in seconds, for example.

Note

Effective only in GUI. Works similar to rest_framework.IntegerField in API.

class vstutils.api.fields.VSTCharField(*args, **kwargs)[source]¶: CharField (extends rest_framework.fields.CharField). This field translate any json type to string for model.

Validators¶

There are validation classes for fields.

class vstutils.api.validators.ImageBaseSizeValidator(extensions=None, **kwargs)[source]¶

Validates image size To use this class for validating image width/height, rewrite self.orientation to (‘height’,) or (‘width’,) or (‘height’, ‘width’)

Raises rest_framework.exceptions.ValidationError: if not(min <= (height or width) <= max)

Parameters: extensions (typing.Union[typing.Tuple, typing.List, None]) –

class vstutils.api.validators.ImageHeightValidator(extensions=None, **kwargs)[source]¶

Wrapper for ImageBaseSizeValidator that validates only height

Parameters

min_height – minimal height of an image being validated
max_height – maximal height of an image being validated
extensions (typing.Union[typing.Tuple, typing.List, None]) –

class vstutils.api.validators.ImageOpenValidator(extensions=None, **kwargs)[source]¶

Image validator that checks if image can be unpacked from b64 to PIL Image obj

Raises rest_framework.exceptions.ValidationError if PIL throws error when trying to open image

Parameters: extensions (typing.Union[typing.Tuple, typing.List, None]) –

class vstutils.api.validators.ImageResolutionValidator(extensions=None, **kwargs)[source]¶

Wrapper for ImageBaseSizeValidator that validates both height and width

Parameters

min_height – minimal height of an image being validated
max_height – maximal height of an image being validated
min_width – minimal width of an image being validated
max_width – maximal width of an image being validated
extensions (typing.Union[typing.Tuple, typing.List, None]) –

class vstutils.api.validators.ImageValidator(extensions=None, **kwargs)[source]¶

Base Image Validation class Validates image format Won’t work if Pillow isn’t installed :type extensions: typing.Union[typing.Tuple, typing.List, None] :param extensions: Tuple or List of file extensions, that should pass the validation Raises rest_framework.exceptions.ValidationError: in case file extension are not in the list

property has_pillow¶: Check if Pillow is installed

class vstutils.api.validators.ImageWidthValidator(extensions=None, **kwargs)[source]¶

Wrapper for ImageBaseSizeValidator that validates only width

Parameters

min_width – minimal width of an image being validated
max_width – maximal width of an image being validated
extensions (typing.Union[typing.Tuple, typing.List, None]) –

class vstutils.api.validators.RegularExpressionValidator(regexp=None)[source]¶

Class for regular expression based validation

Raises: rest_framework.exceptions.ValidationError – in case value does not match regular expression
Parameters: regexp (typing.Optional[typing.Pattern]) –

class vstutils.api.validators.UrlQueryStringValidator(regexp=None)[source]¶

Class for validation url query string, for example a=&b=1

Parameters: regexp (typing.Optional[typing.Pattern]) –

Serializers¶

Default serializer classes for web-api. Read more in Django REST Framework documentation for Serializers.

class vstutils.api.serializers.BaseSerializer(*args, **kwargs)[source]¶: Default serializer with logic to work with objects. Read more in DRF serializer’s documentation how to create Serializers and work with them.

class vstutils.api.serializers.EmptySerializer(*args, **kwargs)[source]¶: Default serializer for empty responses. In generated GUI this means that action button which will not show additional view before execution.

class vstutils.api.serializers.VSTSerializer(*args, **kwargs)[source]¶: Default model serializer based on rest_framework.serializers.ModelSerializer. Read more in DRF documentation how to create Model Serializers. This serializer matches model fields to extended set of serializer fields. List of available pairs specified in VSTSerializer.serializer_field_mapping. For example, to set vstutils.api.fields.FkModelField in serializer use vstutils.models.fields.FkModelField in a model.

Views¶

Default ViewSets for web-api.

class vstutils.api.base.CopyMixin(**kwargs)[source]¶

Mixin for viewsets which adds copy endpoint to view.

copy(request, **kwargs)[source]¶

Endpoint which copy instance with deps.

Parameters: request (rest_framework.request.Request) –
Return type: vstutils.api.responses.BaseResponseClass

copy_field_name = 'name'¶: Name of field which will get a prefix.

copy_prefix = 'copy-'¶: Value of prefix which will be added to new instance name.

copy_related: Iterable[str] = ()¶: List of related names which will be copied to new instance.

class vstutils.api.base.FileResponseRetrieveMixin(**kwargs)[source]¶

ViewSet mixin for retriving FileResponse from models with file fields data.

Example:

import datetime
import os
from django.db import models
from rest_framework import permissions
from vstutils.custom_model import ListModel, FileModel
from vstutils.api import fields, base


class FileViewMixin(base.FileResponseRetrieveMixin):
    # required always
    instance_field_data = 'value'
    # required for response caching in browser
    instance_field_timestamp = 'updated'
    # this is not required, but allow to understand file response mime type in schema
    produces_for_retrieve = ['application/octet-stream', 'application/pdf']
    # search this field in instance for response filename
    # instance_field_filename = 'filename'  # as default
    # headers for response caching (default works with user auth)
    # cache_control_header_data = 'private, no-cache'
    #
    # WARNING:
    # DO NOT OVERRIDE `serializer_class_retrieve` if you do not know what you do.

class ListOfFiles(ListModel):
    data = [
        dict(id=i, value='File data', updated=datetime.datetime(2021, 3, 1, 16, 15, 51, 801564))
        for i in range(1)
    ]
    id = models.IntegerField(primary_key=True)
    value = models.TextField()
    updated = models.DateTimeField()

    class Meta:
        _view_class = (FileViewMixin, 'read_only')
        _list_fields = _detail_fields = ('id', 'filename',)
        _permission_classes = [permissions.AllowAny]
        _override_permission_classes = True

    @property
    def filename(self):
        return f"File_{self.id}.txt"

serializer_class_retrieve¶: alias of django.http.response.FileResponse

class vstutils.api.base.GenericViewSet(**kwargs)[source]¶

The base class for all views. Extends the standard features of the DRF class. Here are some of the possibilities:

Provides model attribute instead of queryset.
Provides to set serializers for each action separately through a dictionary action_serializers or attributes starting with serializer_class_[action name].
Provides to specify a serializer for lists and detail views separately.
Optimizes the database query for GET requests, if possible, by selecting only the fields necessary for the serializer.

get_serializer(*args, **kwargs)[source]¶

Return the serializer instance that should be used for validating and deserializing input, and for serializing output.

Provide to use django.http.StreamingHttpResponse as serializer init.

Parameters

args (typing.Any) –
kwargs (typing.Any) –

Return type

rest_framework.serializers.BaseSerializer

get_serializer_class()[source]¶: Provides to setup serializer class for each action.

nested_allow_check()[source]¶: Just raise or pass. Used for nested views for easier access checking.

class vstutils.api.base.HistoryModelViewSet(**kwargs)[source]¶: Default viewset like ReadOnlyModelViewSet but for historical data (allow to delete, but can’t create and update). Inherited from GenericViewSet.

class vstutils.api.base.ModelViewSet(**kwargs)[source]¶

A viewset that provides CRUD actions under model. Inherited from GenericViewSet.

Variables

model (vstutils.models.BModel) – DB model with data.
serializer_class (vstutils.api.serializers.VSTSerializer) – Serializer for view of Model data.
serializer_class_one (vstutils.api.serializers.VSTSerializer) – Serializer for view one instance of Model data.
serializer_class_[ACTION_NAME] (vstutils.api.serializers.VSTSerializer) – Serializer for view of any endpoint like .create.

Examples:

from vstutils.api.base import ModelViewSet
from . import serializers as sers


class StageViewSet(ModelViewSet):
    # This is difference with DRF:
    # we use model instead of queryset
    model = sers.models.Stage
    # Serializer for list view (view for a list of Model instances
    serializer_class = sers.StageSerializer
    # Serializer for page view (view for one Model instance).
    # This property is not required, if its value is the same as `serializer_class`.
    serializer_class_one = sers.StageSerializer
    # Allowed to set decorator to custom endpoint like this:
    # serializer_class_create - for create method
    # serializer_class_copy - for detail endpoint `copy`.
    # etc...

class vstutils.api.base.ReadOnlyModelViewSet(**kwargs)[source]¶: Default viewset like vstutils.api.base.ModelViewSet for readonly models. Inherited from GenericViewSet.

class vstutils.api.decorators.nested_view(name, arg=None, methods=None, *args, **kwargs)[source]¶

By default DRF does not support nested views. This decorator solves this problem.

You need two or more models with nested relationship (Many-to-Many or Many-to-One) and two viewsets. Decorator nests viewset to parent viewset class and generate paths in API.

Parameters

name (str) – – Name of nested path. Also used as default name for related queryset (see manager_name).
arg (str) – – Name of nested primary key field.
view (vstutils.api.base.ModelViewSet, vstutils.api.base.HistoryModelViewSet, vstutils.api.base.ReadOnlyModelViewSet) – – Nested viewset class.
allow_append (bool) – – Flag for allowing to append existed instances.
manager_name (str) – – Name of model-object attr which contains nested queryset.
methods (list) – – List of allowed methods to nested view endpoints.
subs (list,None) – – List of allowed subviews or actions to nested view endpoints.

Note

Some view methods will not calling for performance reasons. This also applies to some of the class attributes that are usually initialized in the methods. For example, .initial() will never called. Each viewset wrapped by nested class with additional logic.

Example:

from vstutils.api.decorators import nested_view
from vstutils.api.base import ModelViewSet
from . import serializers as sers


class StageViewSet(ModelViewSet):
    model = sers.models.Stage
    serializer_class = sers.StageSerializer


nested_view('stages', 'id', view=StageViewSet)
class TaskViewSet(ModelViewSet):
    model = sers.models.Task
    serializer_class = sers.TaskSerializer

This code generates api paths:

/tasks/ - GET,POST

/tasks/{id}/ - GET,PUT,PATCH,DELETE

/tasks/{id}/stages/ - GET,POST

/tasks/{id}/stages/{stages_id}/ - GET,PUT,PATCH,DELETE

vstutils.api.decorators.subaction(*args, **kwargs)[source]¶

Decorator which wrap object method to subaction of viewset.

Parameters

methods (list) – List of allowed HTTP-request methods. Default is ["post"].
detail (bool) – Flag to set method execution to one instance.
serializer_class (vstutils.api.serializers.VSTSerializer) – Serializer for this action.
permission_classes – Tuple or list permission classes.
url_path (str) – API-path name for this action.
description (str) – Description for this action in OpenAPI.
multiaction (bool) – Allow to use this action in multiactions. Works only with vstutils.api.serializers.EmptySerializer as response.

Filtersets¶

For greater development convenience, the framework provides additional classes and functions for filtering elements by fields.

class vstutils.api.filters.DefaultIDFilter(data=None, queryset=None, *, request=None, prefix=None)[source]¶: Basic filterset to search by id. Provides a search for multiple values separated by commas. Uses extra_filter() in fields.

class vstutils.api.filters.DefaultNameFilter(data=None, queryset=None, *, request=None, prefix=None)[source]¶: Basic filterset to search by part of name. Uses LIKE DB condition by name_filter().

class vstutils.api.filters.FkFilterHandler(related_pk='id', related_name='name', pk_handler=<class 'int'>)[source]¶

Simple handler for filtering by relational fields.

Parameters

related_pk (str) – Field name of related model’s primary key. Default is ‘id’.
related_name (str) – Field name of related model’s charfield. Default is ‘name’.
pk_handler (typing.Callable) – Changes handler for checking value before search. Sends “0” in handler falls. Default is ‘int()’.

Example:

class CustomFilterSet(filters.FilterSet):
    author = CharFilter(method=vst_filters.FkFilterHandler(related_pk='pk', related_name='email'))

Where author is ForeignKey to User and you want to search by primary key and email.

vstutils.api.filters.extra_filter(queryset, field, value)[source]¶

Method for searching values in a comma-separated list.

Parameters

queryset (django.db.models.query.QuerySet) – model queryset for filtration.
field (str) – field name in FilterSet. Also supports __not suffix.
value (str) – comma separated list of searching values.

Returns

filtered queryset.

Return type

django.db.models.query.QuerySet

vstutils.api.filters.name_filter(queryset, field, value)[source]¶

Method for searching by part of name. Uses LIKE DB condition or contains qs-expression.

Parameters

queryset (django.db.models.query.QuerySet) – model queryset for filtration.
field (str) – field name in FilterSet. Also supports __not suffix.
value (str) – searching part of name.

Returns

filtered queryset.

Return type

django.db.models.query.QuerySet

Responses¶

DRF provides a standard set of variables whose names correspond to the human-readable name of the HTTP code. For convenience, we have dynamically wrapped it in a set of classes that have appropriate names and additionally provides following capabilities:

String responses are wrapped in json like { "detail": "string response" }.
Attribute timings are kept for further processing in middlewares.
Status code is set by class name (e.g. HTTP_200_OK or Response200 has code 200).

All classes inherit from:

class vstutils.api.responses.BaseResponseClass(*args, **kwargs)[source]¶

API response class with default status code.

Variables

status_code (int) – HTTP status code.
timings (int,None) – Response timings.

Parameters

timings – Response timings.

Middlewares¶

By default, Django supposes that a developer creates Middleware class manually, but it’s often a routine. The vstutils library offers a convenient request handler class for elegant OOP development. Middleware is used to process incoming requests and send responses before they reach final destination.

class vstutils.middleware.BaseMiddleware(get_response)[source]¶

Middleware base class for handling:

Incoming requests by BaseMiddleware.request_handler();
Outgoing response before any calling on server by BaseMiddleware.get_response_handler();
Outgoing responses by BaseMiddleware.handler().

Middleware must be added to MIDDLEWARE list in settings.

Example:

from vstutils.middleware import BaseMiddleware
from django.http import HttpResponse


class CustomMiddleware(BaseMiddleware):
    def request_handler(self, request):
        # Add header to request
        request.headers['User-Agent'] = 'Mozilla/5.0'
        return request

    def get_response_handler(self, request):
        if not request.user.is_stuff:
            # Return 403 HTTP status for non-stuff users.
            # This request never gets in any view.
            return HttpResponse(
                "Access denied!",
                content_type="text/plain",
                status_code=403
            )
        return super().get_response_handler(request)

    def handler(self, request, response):
        # Add header to response
        response['Custom-Header'] = 'Some value'
        return response

Parameters: get_response (typing.Callable) –

get_response_handler(request)[source]¶

Entrypoint for breaking or continuing request handling. This function must return django.http.HttpResponse object or result of parent class calling.

Parameters: request (django.http.HttpRequest) – HTTP-request object which is wrapped from client request.
Return type: django.http.HttpResponse

handler(request, response)[source]¶

The response handler. Method to process responses.

Parameters

request (django.http.HttpRequest) – HTTP-request object.
response (django.http.HttpResponse) – HTTP-response object which will be sended to client.

Returns

Handled response object.

Return type

django.http.HttpResponse

request_handler(request)[source]¶

The request handler. Called before request is handled by a view.

Parameters: request (django.http.HttpRequest) – HTTP-request object which is wrapped from client request.
Returns: Handled request object.
Return type: django.http.HttpRequest

Filter Backends¶

Filter Backends are used to modify model queryset. To create custom filter backend to, (i.g. annotate model queryset), you should inherit from :class:vstutils.api.filter_backends.VSTFilterBackend and override :meth:vstutils.api.filter_backends.VSTFilterBackend.filter_queryset and in some cases :meth:vstutils.api.filter_backends.VSTFilterBackend.get_schema_fields.

class vstutils.api.filter_backends.DeepViewFilterBackend[source]¶

Backend that filters queryset by column from deep_parent_field property of the model. Value for filtering must be provided in query param __deep_parent.

If param is missing then no filtering is applied.

If param is empty value (/?__deep_parent=) then objects with no parent (the value of the field whose name is stored in the property deep_parent_field of the model is None) returned.

This filter backend and nested view is automatically added when model has deep_parent_field property.

Example:

from django.db import models
from vstutils.models import BModel

class DeepNestedModel(BModel):
    name = models.CharField(max_length=10)
    parent = models.ForeignKey('self', null=True, default=None, on_delete=models.CASCADE)

    deep_parent_field = 'parent'
    deep_parent_allow_append = True

    class Meta:
        default_related_name = 'deepnested'

In example above if we add this model under path ‘deep’, following views will be created: /deep/ and /deep/{id}/deepnested/.

Filter backend can be used as /deep/?__deep_parent=1 and will return all DeepNestedModel objects whose parent’s primary key is 1.

You can also use generic DRF views, for that you still must set deep_parent_field to your model and manually add DeepViewFilterBackend to filter_backends list.

class vstutils.api.filter_backends.HideHiddenFilterBackend[source]¶

Filter Backend that hides all objects with hidden=True from the queryset

filter_queryset(request, queryset, view)[source]¶: Clear objects with hidden attr from queryset.

class vstutils.api.filter_backends.SelectRelatedFilterBackend[source]¶

Filter Backend that will automatically call prefetch_related and select_related on all relations in queryset.

filter_queryset(request, queryset, view)[source]¶: Select related in queryset.

class vstutils.api.filter_backends.VSTFilterBackend[source]¶

A base filter backend class to be inherited from. Example:

from django.utils import timezone
from django.db.models import Value, DateTimeField

from vstutils.api.filter_backends import VSTFilterBackend

class CurrentTimeFilterBackend(VSTFilterBackend):
    def filter_queryset(self, request, queryset, view):
        return queryset.annotate(current_time=Value(timezone.now(), output_field=DateTimeField()))

In this example Filter Backend annotates time in current timezone to any connected model’s queryset.

In some cases it may be necessary to provide a parameter from a query of request. To define this parameter in the schema, you must overload the get_schema_fields function and specify a list of parameters to use.

Example:

from django.utils import timezone
from django.db.models import Value, DateTimeField

from vstutils.api.filter_backends import VSTFilterBackend

class ConstantCurrentTimeForQueryFilterBackend(VSTFilterBackend):
    query_param = 'constant'

    def filter_queryset(self, request, queryset, view):
        if self.query_param in request.query_params and request.query_params[self.query_param]:
            queryset = queryset.annotate(**{
                request.query_params[self.query_param]: Value(timezone.now(), output_field=DateTimeField())
            })
        return queryset

        def get_schema_fields(self, view):
            return [
                compat.coreapi.Field(
                    name=self.query_param,
                    required=False,
                    location='query',
                    schema=compat.coreschema.String(
                        description='Annotate value to queryset'
                    ),
                )
            ]

In this example Filter Backend annotates time in current timezone to any connected model’s queryset with field name from query constant.

get_schema_fields(view)[source]¶: You can also make the filter controls available to the schema autogeneration that REST framework provides, by implementing this method. The method should return a list of coreapi.Field instances.

Celery¶

Celery is a distributed task queue. It’s used to execute some actions asynchronously in a separate worker. For more details on Celery, check it’s official docs. For Celery related vstutils features to work, you need to specify [rpc] and [worker] sections in settings.ini. Also you need to include extra [rpc] requirements.

Tasks¶

class vstutils.tasks.TaskClass[source]¶

Wrapper for Celery BaseTask class. Usage is same as Celery standard class, but you can execute task without creating instance with TaskClass.do() method.

Example:

from vstutils.environment import get_celery_app
from vstutils.tasks import TaskClass

app = get_celery_app()

class Foo(TaskClass):
    def run(*args, **kwargs):
        return 'Foo task has been executed'

app.tasks.register(Foo())

Now you can call your task with various methods:

by executing Foo.do(*args, **kwargs)
get registered task instance like that - app.tasks[‘full_path.to.task.class.Foo’]

Also you can make you registered task periodic, by adding it to CELERY_BEAT_SCHEDULE in settings.py:

CELERY_BEAT_SCHEDULE = {
    'foo-execute-every-month': {
        'task': 'full_path.to.task.class.Foo',
        'schedule': crontab(day_of_month=1),
    },
}

classmethod do(*args, **kwargs)[source]¶

Method which send signal to celery for start remote task execution. All arguments will passed to the task TaskClass.run() method.

Return type: celery.result.AsyncResult

property name¶: property for proper Celery task execution, needed for TaskClass.do() method to work

Endpoint¶

Endpoint view has two purposes: bulk requests execution and providing openapi schema.

Endpoint url is /{API_URL}/endpoint/, for example value with default settings is /api/endpoint/.

API_URL can be changed in settings.py.

class vstutils.api.endpoint.EndpointViewSet(**kwargs)[source]¶

Default API-endpoint viewset.

get(request)[source]¶

Returns response with swagger ui or openapi json schema if ?format=openapi

Parameters: request (vstutils.api.endpoint.BulkRequestType) –
Return type: django.http.response.HttpResponse

get_client(request)[source]¶

Returns test client and guarantees that if bulk request comes authenticated than test client will be authenticated with the same user

Parameters: request (vstutils.api.endpoint.BulkRequestType) –
Return type: vstutils.api.endpoint.BulkClient

get_serializer(*args, **kwargs)[source]¶

Return the serializer instance that should be used for validating and deserializing input, and for serializing output.

Return type: vstutils.api.endpoint.OperationSerializer

get_serializer_context(context)[source]¶

Extra context provided to the serializer class.

Return type: dict

operate(operation_data, context)[source]¶

Method used to handle one operation and return result of it

Parameters

operation_data (typing.Dict) –
context (typing.Dict) –

Return type

typing.Tuple[typing.Dict, typing.SupportsFloat]

post(request)[source]¶

Execute transactional bulk request

Parameters: request (vstutils.api.endpoint.BulkRequestType) –
Return type: vstutils.api.responses.BaseResponseClass

put(request, allow_fail=True)[source]¶

Execute non transaction bulk request

Parameters: request (vstutils.api.endpoint.BulkRequestType) –
Return type: vstutils.api.responses.BaseResponseClass

serializer_class¶

One operation serializer class.

alias of vstutils.api.endpoint.OperationSerializer

versioning_class¶: alias of rest_framework.versioning.QueryParameterVersioning

Bulk requests¶

Bulk request allows you send multiple reques`t to api at once, it accepts json list of operations.

Method	Transactional (all operations in one transaction)	Synchronous (operations executed one by one in given order)
`PUT /{API_URL}/endpoint/`	NO	YES
`POST /{API_URL}/endpoint/`	YES	YES
`PATCH /{API_URL}/endpoint/`	NO	NO

Parameters of one operation (required parameter marked by ^*):

method^* - http method of request
path^* - path of request, can be str or list
data - data to send
query - query parameters as str
headers - dict with headers which will be sent, names of headers must follow CGI specification (e.g., CONTENT_TYPE, GATEWAY_INTERFACE, HTTP_*).
version - str with specified version of api, if not provided then VST_API_VERSION will be used

In any request parameter you can insert result value of previous operations (<<{OPERATION_NUMBER}[path][to][value]>>), for example:

[
    {"method": "post", "path": "user", "data": {"name": "User 1"}),
    {"method": "delete", "version": "v2", "path": ["user", "<<0[data][id]>>"]}
]

Result of bulk request is json list of objects for operation:

method - http method
path - path of request, always str
data - data that needs to be sent
status - response status code

Transactional bulk request returns 502 BAG GATEWAY and does rollback after first failed request.

Warning

If you send non-transactional bulk request, you will get 200 status and must validate statuses on each operation responses.

Openapi schema¶

Request on GET /{API_URL}/endpoint/ returns Swagger UI.

Request on GET /{API_URL}/endpoint/?format=openapi returns openapi schema in json format. Also you can specify required version of schema using version query parameter (e.g., GET /{API_URL}/endpoint/?format=openapi&version=v2).

To change the schema after generating and before sending to user use hooks. Define one or more function, each taking 2 named arguments:

request - user request object.
schema - ordered dict with openapi schema.

Note

Sometimes hooks may raise an exception; in order to keep a chain of data modification, such exceptions are handled. The changes made to the schema before the exception however, are saved.

Example hook:

def hook_add_username_to_guiname(request, schema):
    schema['info']['title'] = f"{request.username} - {schema['info']['title']}"

To connect hook(s) to your app add function import name to the OPENAPI_HOOKS list in settings.py

OPENAPI_HOOKS = [
    '{{appName}}.openapi.hook_add_username_to_guiname',
]

Testing Framework¶

VST Utils Framework includes a helper in base testcase class and improves support for making API requests. That means if you want make bulk request to endpoint you dont need create and init test client, but just need to call:

endpoint_results = self.bulk([
    # list of endpoint requests
])

Creating test case¶

test.py module contains testcase classes based on vstutils.tests.BaseTestCase. At the moment, we officially support two styles of writing tests: classic and simple query wrappers with run check and runtime optimized bulk queries with manual value checking.

Simple example with classic tests¶

For example, if you have api endpoint like /api/v1/project/ and model Project you can write testcase like this:

from vstutils.tests import BaseTestCase


class ProjectTestCase(BaseTestCase):
    def setUp(self):
        super(ProjectTestCase, self).setUp()
        # init demo project
        self.initial_project = self.get_model_class('project.Test').objects.create(name="Test")

    def tearDown(self)
        super(ProjectTestCase, self).tearDown()
        # remove it after test
        self.initial_project.delete()

    def test_project_endpoint(self):
        # Test checks that api returns valid values
        self.list_test('/api/v1/project/', 1)
        self.details_test(
            ["project", self.initial_project.id],
            name=self.initial_project.name
        )
        # Try to create new projects and check list endpoint
        test_data = [
            {"name": f"TestProject{i}"}
            for i in range(2)
        ]
        id_list = self.mass_create("/api/v1/project/", test_data, 'name')
        self.list_test('/api/v1/project/', 1 + len(id_list))

This example demonstrates functionality of default test case class. Default projects are initialized for the fastest and most efficient result. We recommend to divide tests for different entities into different classes. This example demonstrate classic style of testing, but you can use bulks in your test cases.

Bulk requests in tests¶

Bulk query system is well suited for testing and executing valid queries. Previous example could be rewritten as follows:

from vstutils.tests import BaseTestCase


class ProjectTestCase(BaseTestCase):
    def setUp(self):
        super(ProjectTestCase, self).setUp()
        # init demo project
        self.initial_project = self.get_model_class('project.Test').objects.create(name="Test")

    def tearDown(self)
        super(ProjectTestCase, self).tearDown()
        # remove it after test
        self.initial_project.delete()

    def test_project_endpoint(self):
        test_data = [
            {"name": f"TestProject{i}"}
            for i in range(2)
        ]
        bulk_data = [
            {"method": "get", "path": ["project"]},
            {"method": "get", "path": ["project", self.initial_project.id]}
        ]
        bulk_data += [
            {"method": "post", "path": ["project"], "data": i}
            for i in test_data
        ]
        bulk_data.append(
            {"method": "get", "path": ["project"]}
        )
        results = self.bulk_transactional(bulk_data)

        self.assertEqual(results[0]['status'], 200)
        self.assertEqual(results[0]['data']['count'], 1)
        self.assertEqual(results[1]['status'], 200)
        self.assertEqual(results[1]['data']['name'], self.initial_project.name)

        for pos, result in enumerate(results[2:-1]):
            self.assertEqual(result['status'], 201)
            self.assertEqual(result['data']['name'], test_data[pos]['name'])

        self.assertEqual(results[-1]['status'], 200)
        self.assertEqual(results[-1]['data']['count'], 1 + len(test_data))

In this case, you have more code, but your tests are closer to GUI workflow, because vstutils-projects uses /api/endpoint/ for requests. Either way, bulk queries are much faster due to optimization; Testcase execution time is less comparing to non-bulk requests.

Test case API¶

class vstutils.tests.BaseTestCase(methodName='runTest')[source]¶

Main testcase class extends django.test.TestCase.

assertCheckDict(first, second, msg=None)[source]¶

Fail if the two fields in dicts are unequal as determined by the ‘==’ operator. Checks if first not contains or not equal field in second

Parameters

first (typing.Dict) –
second (typing.Dict) –
msg (typing.Optional[str]) –

assertCount(iterable, count, msg=None)[source]¶

Call len() over iterable and check equals with count.

Parameters

iterable (typing.Sized) – any iterable object which could be sended to len().
count (int) – expected result.
msg (typing.Optional[typing.Any]) – error message

assertRCode(resp, code=200, *additional_info)[source]¶

Fail if response code is not equal. Message is response body.

Parameters

resp (django.http.HttpResponse) – response object
code (int) – expected code

bulk(data, code=200, **kwargs)[source]¶

Make non transactional bulk request and assert status code (default is 200)

Parameters

data (typing.Union[typing.List[typing.Dict[str, typing.Any]], str]) – request data
code (int) – http status to assert
kwargs – named arguments for get_result()

Return type

typing.Union[typing.List[typing.Dict[str, typing.Any]], str, typing.Dict, typing.Sequence[typing.Union[typing.List[typing.Dict[str, typing.Any]], str]]]

Returns

bulk response

bulk_transactional(data, code=200, **kwargs)[source]¶

Make transactional bulk request and assert status code (default is 200)

Parameters

data (typing.Union[typing.List[typing.Dict[str, typing.Any]], str]) – request data
code (int) – http status to assert
kwargs – named arguments for get_result()

Return type

typing.Union[typing.List[typing.Dict[str, typing.Any]], str, typing.Dict, typing.Sequence[typing.Union[typing.List[typing.Dict[str, typing.Any]], str]]]

Returns

bulk response

call_registration(data, **kwargs)[source]¶

Function for calling registration. Just got form data and headers.

Parameters

data (dict) – Registration form data.
kwargs – named arguments with request headers.

details_test(url, **kwargs)[source]¶

Test for get details of model. If you setup additional named arguments, the method check their equality with response data. Uses get_result() method.

Parameters

url – url to detail record. For example: /api/v1/project/1/ (where 1 is uniq id of project). You can use get_url() for building url.
kwargs – params that’s should be checked (key - field name, value - field value).

endpoint_call(data=None, method='get', code=200, **kwargs)[source]¶

Make request to endpoint and assert response status code if specified (default is 200). Uses get_result() method for execution.

Parameters

data (typing.Union[typing.List[typing.Dict[str, typing.Any]], str, None]) – request data
method (str) – http request method
code (int) – http status to assert
query – dict with query data (working only with get)

Return type

typing.Union[typing.List[typing.Dict[str, typing.Any]], str, typing.Dict, typing.Sequence[typing.Union[typing.List[typing.Dict[str, typing.Any]], str]]]

Returns

bulk response

endpoint_schema(**kwargs)[source]¶

Make request to schema. Returns dict with swagger data.

Parameters: version – API version for schema parser.

get_count(model, **kwargs)[source]¶

Simple wrapper over get_model_filter() which returns counter of items.

Parameters

model (str,django.db.models.Model) – string which contains model name (if attribute model is set to the testcase class), module import, app.ModelName or django.db.models.Model.
kwargs – named arguments to django.db.models.query.QuerySet.filter().

Returns

number of instances in database.

Return type

int

get_model_class(model)[source]¶

Getting model class by string or return model arg.

Parameters: model (str,django.db.models.Model) – string which contains model name (if attribute model is set to the testcase class), module import, app.ModelName or django.db.models.Model.
Returns: Model class.
Return type: django.db.models.Model

get_model_filter(model, **kwargs)[source]¶

Simple wrapper over get_model_class() which returns filtered queryset from model.

Parameters

model (str,django.db.models.Model) – string which contains model name (if attribute model is set to the testcase class), module import, app.ModelName or django.db.models.Model.
kwargs – named arguments to django.db.models.query.QuerySet.filter().

Return type

django.db.models.query.QuerySet

get_result(rtype, url, code=None, *args, **kwargs)[source]¶

Execute and test response code on request with returning parsed result of request. The method uses the following procedure:

Test client authorization (with user which creates in setUp()).
Executing a request (sending args and kwargs to request method).
Parsing the result (converts json string to python-object).
Checking the http status code with assertRCode() (if you have not specified it, the code will be selected in accordance with the request method from the standard set std_codes).
Logout client.
Return parsed result.

Parameters

rtype – request type (methods from Client cls): get, post etc.
url – requested url string or tuple for get_url(). You can use get_url() for url building or setup it as full string.
code (typing.Optional[int]) – expected return code from request.
relogin – execute force login and logout on each call. Default is True.
args – extra-args for Client class request method.
kwargs – extra-kwargs for Client class request method.

Return type

typing.Union[typing.List[typing.Dict[str, typing.Any]], str, typing.Dict, typing.Sequence[typing.Union[typing.List[typing.Dict[str, typing.Any]], str]]]

Returns

result of request.

get_url(*items)[source]¶

Function for creating url path based on VST_API_URL and VST_API_VERSION settings. Without arguments returns path to default version of api.

Return type: str
Returns: string like /api/v1/.../.../ where ... is args of function.

list_test(url, count)[source]¶

Test for get list of models. Checks only list count. Uses get_result() method.

Parameters

url – url to abstract layer. For example: /api/v1/project/. You can use get_url() for building url.
count – count of objects in DB.

mass_create(url, data, *fields, **kwargs)[source]¶

Mass creation objects in api-abstration. Uses get_result() method.

Parameters

url – url to abstract layer like argument in get_result(). For example: /api/v1/project/.
data – list with data to send on creation and feature checks.
kwargs – extra-kwargs for request method.

Params fields

list of fields to check after creation.

Returns

list of id from all resulted objects.

Return type

list

Note

The method does not use endpoint requests. That mean if you want send some extra headers or data you can do it by additional kwargs.

models = None¶: Attribute with default project models module.

classmethod patch(*args, **kwargs)[source]¶

Simple unittest.mock.patch() class-method wrapper.

Return type: typing.ContextManager[unittest.mock.Mock]

random_name()[source]¶

Simple function which returns uuid1 string.

Return type: str

std_codes: Dict[str, int] = {'delete': 204, 'get': 200, 'patch': 200, 'post': 201}¶: Default http status codes for different http methods. Uses in get_result()

class user_as(testcase, user)[source]¶

Context for execute bulk or something as user. The context manager overrides self.user in TestCase and revert this changes on exit.

Parameters: user (django.contrib.auth.models.AbstractUser) – new user object for execution.

Utils¶

This is tested set of development utilities. Utilities include a collection of code that will be useful in one way or another for developing the application. Vstutils uses mosts of these functions under the hood.

class vstutils.utils.BaseEnum(value)[source]¶

BaseEnum extends Enum class and used to create enum-like objects that can be used in django serializers or django models.

Example:

from vstutils.models import BModel

class ItemCLasses(BaseEnum):
    FIRST='FIRST'
    SECOND='SECOND'
    THIRD='THIRD'


class MyDjangoModel(BModel):
    item_class = models.CharField(max_length=1024, choices=ItemCLasses.to_choices())

    @property
    def is_second(self):
        # Function check is item has second class of instance
        return ItemCLasses.SECOND.is_equal(self.item_class)

class vstutils.utils.BaseVstObject[source]¶

Default mixin-class for custom objects which needed to get settings and cache.

classmethod get_django_settings(name, default=None)[source]¶

Get params from Django settings.

Parameters

name (str) – name of param
default (object) – default value of param

Returns

Param from Django settings or default.

class vstutils.utils.Dict[source]¶: Wrapper over dict which return JSON on conversion to string.

class vstutils.utils.Executor(stdout=- 1, stderr=- 2, **environ_variables)[source]¶

Command executor with realtime output write and line handling. By default and by design executor initialize string attribute output which will be modifyed by += operator with new lines by Executor.write_output() procedure. Override the method if you want change behavior.

Executor class supports periodically (0.01 sec) handling process and execute some checks by overriding Executor.working_handler() procedure method. If you want disable this behavior override the method by None value or use UnhandledExecutor.

exception CalledProcessError(returncode, cmd, output=None, stderr=None)¶

Raised when run() is called with check=True and the process returns a non-zero exit status.

Attributes:: cmd, returncode, stdout, stderr, output

property stdout¶: Alias for output attribute, to match stderr

execute(cmd, cwd)[source]¶

Execute commands and output this.

Parameters

cmd (list) – – list of cmd command and arguments
cwd (str) – – workdir for executions

Returns

– string with full output

Return type

str

working_handler(proc)[source]¶

Additional handler for executions.

Parameters: proc (subprocess.Popen) –
Return type: None

write_output(line)[source]¶

Parameters: line (str) – – line from command output
Returns: None
Return type: None

class vstutils.utils.KVExchanger(key, timeout=None)[source]¶: Class for transmit data using key-value fast (cache-like) storage between services. Uses same cache-backend as Lock.

class vstutils.utils.Lock(id, payload=1, repeat=1, err_msg='', timeout=None)[source]¶

Lock class for multi-jobs workflow. Based on KVExchanger. The Lock allows ony one thread to enter the part that’s locked and shared between apps using one locks cache (see also [locks]).

Parameters

id (int,str) – – unique id for lock.
payload – – lock additional info. Should be any boolean True value.
repeat (int) – – time to wait lock.release. Default 1 sec.
err_msg (str) – – message for AcquireLockException error.

Note

Used django.core.cache lib and settings in settings.py
Have Lock.SCHEDULER and Lock.GLOBAL id

Example:

from vstutils.utils import Lock

with Lock("some_lock_identifier", repeat=30, err_msg="Locked by another proccess") as lock:
    # where
    # ``"some_lock_identifier"`` is unique id for lock and
    # ``30`` seconds lock is going wait until another process will release lock id.
    # After 30 sec waiting lock will raised with :class:`.Lock.AcquireLockException`
    # and ``err_msg`` value as text.
    some_code_execution()
    # ``lock`` object will has been automatically released after
    # exiting from context.

Another example without context manager:

from vstutils.utils import Lock

# locked block after locked object created
lock = Lock("some_lock_identifier", repeat=30, err_msg="Locked by another proccess")
# deleting of object calls ``lock.release()`` which release and remove lock from id.
del lock

exception AcquireLockException[source]¶: Exception which will raised on unreleased lock.

class vstutils.utils.ModelHandlers(type_name, err_message=None)[source]¶

Handlers for some models like ‘INTEGRATIONS’ or ‘REPO_BACKENDS’. Based on ObjectHandlers but more specific for working with models. All handlers backends get by first argument model object.

Attributes:

Parameters

objects (dict) – – dict of objects like: {<name>: <backend_class>}
keys (list) – – names of supported backends
values (list) – – supported backends classes
type_name (str) –
err_message (typing.Optional[str]) –

get_object(name, obj)[source]¶

Parameters

name (str) – – string name of backend
name – str
obj (django.db.models.Model) – – model object

Returns

backend object

Return type

object

class vstutils.utils.ObjectHandlers(type_name, err_message=None)[source]¶

Handlers wrapper for get objects from some settings structure.

Example:

from vstutils.utils import ObjectHandlers

'''
In `settings.py` you should write some structure:

SOME_HANDLERS = {
    "one": {
        "BACKEND": "full.python.path.to.module.SomeClass"
    },
    "two": {
        "BACKEND": "full.python.path.to.module.SomeAnotherClass",
        "OPTIONS": {
            "some_named_arg": "value"
        }
    }
}
'''

handlers = ObjectHandlers('SOME_HANDLERS')

# Get class handler for 'one'
one_backend_class = handlers['one']
# Get object of backend 'two'
two_obj = handlers.get_object()
# Get object of backend 'two' with overriding constructor named arg
two_obj_overrided = handlers.get_object(some_named_arg='another_value')

Parameters

type_name (str) –
err_message (typing.Optional[str]) –

backend(name)[source]¶

Get backend class

Parameters: name (str) – – name of backend type
Returns: class of backend
Return type: type,types.ModuleType,object

class vstutils.utils.Paginator(qs, chunk_size=None)[source]¶: Class for fragmenting the query for small queries.

class vstutils.utils.SecurePickling(secure_key=None)[source]¶

Secured pickle wrapper by Vigenère cipher.

Warning

Do not use it with untrusted transport anyway.

Example:

from vstutils.utils import SecurePickling


serializer = SecurePickling('password')

# Init secret object
a = {"key": "value"}
# Serialize object with secret key
pickled = serializer.dumps(a)
# Deserialize object
unpickled = serializer.loads(pickled)

# Check, that object is correct
assert a == unpickled

Parameters: secure_key (typing.Optional[str]) –

class vstutils.utils.URLHandlers(type_name='URLS', *args, **kwargs)[source]¶

Object handler for GUI views. Uses GUI_VIEWS from settings.py. Based on ObjectHandlers but more specific to urlpatterns.

Example:

from vstutils.utils import URLHandlers


# By default gets from `GUI_VIEWS` in `settings.py`
urlpatterns = list(URLHandlers())

Parameters: type_name (str) –

get_object(name, *argv, **kwargs)[source]¶

Get url object tuple for urls.py

Parameters

name (str) – url regexp from
argv – overrided args
kwargs – overrided kwargs

Returns

url object

Return type

django.urls.re_path

class vstutils.utils.UnhandledExecutor(stdout=- 1, stderr=- 2, **environ_variables)[source]¶: Class based on Executor but disables working_handler.

class vstutils.utils.apply_decorators(*decorators)[source]¶

Decorator which apply list of decorators on method or class.

Example:

from vstutils.utils import apply_decorators

def decorator_one(func):
    print(f"Decorated {func.__name__} by first decorator.")
    return func

def decorator_two(func):
    print(f"Decorated {func.__name__} by second decorator.")
    return func

@apply_decorators(decorator_one, decorator_two)
def decorated_function():
    # Function decorated by both decorators.
    print("Function call.")

class vstutils.utils.classproperty(fget, fset=None)[source]¶

Decorator which makes class method as class property.

Example:

from vstutils.utils import classproperty

class SomeClass(metaclass=classproperty.meta):
    # Metaclass is needed for set attrs in class
    # instead of and not only object.

    some_value = None

    @classproperty
    def value(cls):
        return cls.some_value

    @value.setter
    def value(cls, new_value):
        cls.some_value = new_value

Parameters

fget (typing.Callable) –
fset (typing.Optional[typing.Callable]) –

vstutils.utils.decode(key, enc)[source]¶

Decode string from encoded by Vigenère cipher.

Parameters

key (str) – – secret key for encoding
enc (str) – – encoded string for decoding

Returns

– decoded string

Return type

str

vstutils.utils.deprecated(func)[source]¶

This is a decorator which can be used to mark functions as deprecated. It will result in a warning being emitted when the function is used.

Parameters: func (typing.Callable) –

vstutils.utils.encode(key, clear)[source]¶

Encode string by Vigenère cipher.

Parameters

key (str) – – secret key for encoding
clear (str) – – clear value for encoding

Returns

– encoded string

Return type

str

vstutils.utils.get_render(name, data, trans='en')[source]¶

Render string from template.

Parameters

name (str) – – full template name
data (dict) – – dict of rendered vars
trans (str) – – translation for render. Default ‘en’.

Returns

– rendered string

Return type

str

vstutils.utils.list_to_choices(items_list, response_type=<class 'list'>)[source]¶

Method to create django model choices from flat list of values.

Parameters

items_list (typing.Iterable) – list of flat values.
response_type (typing.Callable) – casting type of returned mapping

Return type

typing.Iterable[typing.Tuple[str, str]]

Returns

list of tuples from items_list values

class vstutils.utils.model_lock_decorator(**kwargs)[source]¶

Decorator for functions where ‘pk’ kwarg exist for lock by id.

Warning

On locked error raised Lock.AcquireLockException
Method must have and called with pk named arg.

class vstutils.utils.raise_context(*args, **kwargs)[source]¶: Context for exclude exceptions.

class vstutils.utils.raise_context_decorator_with_default(*args, **kwargs)[source]¶

Context for exclude errors and return default value.

Example:

from yaml import load
from vstutils.utils import raise_context_decorator_with_default


@raise_context_decorator_with_default(default={})
def get_host_data(yaml_path, host):
    with open(yaml_path, 'r') as fd:
        data = load(fd.read(), Loader=Loader)
    return data[host]
    # This decorator used when you must return some value even on error
    # In log you also can see traceback for error if it occur

def clone_host_data(host):
    bs_data = get_host_data('inventories/aws/hosts.yml', 'build_server')
    ...

class vstutils.utils.redirect_stdany(new_stream=<_io.StringIO object>, streams=None)[source]¶

Context for redirect any output to own stream.

Note

On context return stream object.
On exit return old streams

vstutils.utils.send_template_email(sync=False, **kwargs)[source]¶

Function executing sync or async email sending; according sync argument and settings variable “RPC_ENABLED”. You can use this function to send message, it sends message asynchronously or synchronously. If you don’t set settings for celery or don’t have celery it sends synchronously mail. If celery is installed and configured and sync argument of the function is set to False, it sends asynchronously email.

Parameters

sync (bool) – argument for determining how send email, asynchronously or synchronously
subject – mail subject.
email – list of strings or single string, with email addresses of recipients
template_name – relative path to template in templates directory, must include extension in file name.
context_data – dictionary with context for rendering message template.

vstutils.utils.send_template_email_handler(subject, email_from, email, template_name, context_data=None, **kwargs)[source]¶

Function for email sending. The function convert recipient to list and set context before sending if it possible.

Parameters

subject (str) – mail subject.
email_from (str) – sender that be setup in email.
email (typing.Union[typing.List, str]) – list of strings or single string, with email addresses of recipients
template_name (str) – relative path to template in templates directory, must include extension in file name.
context_data (typing.Optional[typing.Dict]) – dictionary with context for rendering message template.
kwargs – additional named arguments for send_mail

Return type

typing.SupportsInt

Returns

Number of emails sent.

class vstutils.utils.tmp_file(data='', mode='w', bufsize=- 1, **kwargs)[source]¶

Temporary file with name generated and auto removed on close.

Attributes:

Parameters

data (str) – – string to write in tmp file.
mode (str) – – file open mode. Default ‘w’.
bufsize (int) – – bufer size for tempfile.NamedTemporaryFile
kwargs – – other kwargs for tempfile.NamedTemporaryFile

write(wr_string)[source]¶

Write to file and flush

Parameters: wr_string (str) – – writable string
Returns: None
Return type: None

class vstutils.utils.tmp_file_context(*args, **kwargs)[source]¶

Context object for work with tmp_file. Auto close on exit from context and remove if file still exist.

This context manager over tmp_file

VST Utils documentation

Backend API manual¶

Models¶

You can also use custom models without using database:¶

Model Fields¶

Web API¶

Fields¶

Validators¶

Serializers¶

Views¶

Filtersets¶

Responses¶

Middlewares¶

Filter Backends¶

Celery¶

Tasks¶

Endpoint¶

Bulk requests¶

Openapi schema¶

Testing Framework¶

Creating test case¶

Simple example with classic tests¶

Bulk requests in tests¶

Test case API¶

Utils¶