I have a custom workflow where on button click I open a specific primary form-view which has "js_class" attribute (the returned view will have custom JS behaviour). 

The returned view will also be opened a Dialog, not fullscreen, so it will have both behaviour from FormView widget and FormViewDialog widget. 

I can customize the FormController behaviour of CustomFormView by mapping my customFormController in the config property of my CustomFormView:

var customFormController = FormController.extend({
//..... custom controller behaviour
});

var CustomFormView = FormView.extend({
    config: _.extend({}, FormView.prototype.config, {
        Controller: customFormController, // map custom controller to the form-view
    });

Now I would like to make an extension for Dialog because I want to extend some behaviours which belongs to FormViewDialog widget

var dialogs = require('web.view_dialogs');

var customFormDialog = dialogs.FormViewDialog.extend({
// ... custom dialog behaviour for the formview
});

It is possible to map customFormDialog as well to my js_class view somehow, so that when the form-view is opened in a FormViewDialog widget it will use my customFormDialog extension instead of the original FormViewDialog widget?

I think I found the connection between Form Controller and FormViewDialog: I see that the FormViewDialog render is managed by _onOpenRecord event ( _onOpenOne2ManyRecord in case of click on o2m fields). Here you could actually decide to make an evaluation and choose which extension should be opened, but I am not used to js and these two events does not seem to be very extendable, so I was looking for an easier and cleaner solution, if possible.