Security

Overview

The platform implements multiple mechanisms to secure access to resources and keep your data safe. All user and security management is done under a single pane of glass: the configuration is done in one place, in the user-friendly graphical platform dashboard, and applied across all platform interfaces. The result is a significantly simplified, yet robust, data-security solution that helps organizations meet their compliance objectives.

The platform allows you to define local users and import users from an external identity provider (IdP), authenticate user identities, and control users' access to resources, including the ability to define fine-grained data-access policies. To ensure proper security, the platform uses time-limited sessions and supports the HTTP Secure (HTTPS) protocol. You can view logs for security events and user actions (such as a failed login or deletion of a data container) on the Events | Audit dashboard tab.

HTTP Secure Data Transmission

For enhanced security, the platform's RESTful web and cluster-management APIs support the HTTP Secure (HTTPS) protocol (also known as HTTP over TLS), as defined in the RFC 2818 specification. See HTTPS Requests in the Securing Your Web-API Requests reference.

Authentication

Authentication is the process of validating a user's identity before granting the user access to a specific resource. Before granting a user access to resources, the platform verifies (authenticates) the identity of the user and then ensures that the user has the required permissions to perform the requested operation (see Authorization). To support authentication, the platform uses time-limited sessions and access keys. The time-to-live (TTL) session period is 24 hours.

Authentication of data-access requests is done using data sessions, which are handled transparently by the platform.

Authentication of management requests is done using management sessions, which are created transparently when performing operations from the platform dashboard, or handled by the user using the platform's RESTful cluster-management APIs ("the management APIs") [Beta]. These APIs use a session-based HTTP scheme to support user authentication and authorization: access to management API resources requires a time-limited session cookie that is used to authenticate the sender of the request with a username and password, and determine the sender's authorization to perform the requested operation. See the Overview [Beta].

In addition, the platform's web APIs support user authentication by using either the username/password Basic HTTP authentication scheme or custom access-key (session-key) authentication. With either method, the user provides an authentication header with user credentials that are verified by the platform as a condition for sending the request. See HTTP User Authentication in the Securing Your Web-API Requests reference.

Access keys

Access keys are not time-bound, and are used mainly for programmatic purposes. Access keys are created by authenticated users and can allow only the actions in the scope of the user creating the key: each specific resource access is defined by the management policies of the user.

Access keys are specific to either the data plane, the control plane, or to both. The data plane is relevant to operations on the data itself (put object, write to stream, read kv etc.). The control plane includes create, read, update, and delete, operations on all resources that aren't specifically data in the containers, for example: projects, users, groups, management policies, the containers themselves (not the data inside them), services, etc.

Access keys are not limited to a specific use or environment. They can be delegated and passed on to other entities. The clients running inside Jupyter use the access key from Jupyter and pass it to the resources they create. For example, when MLRun SDK is running inside Jupyter, it passes the access key to the MLRun jobs it creates.

Since access keys be used by anyone who has access to them, they should be closely watched, and their usage should be well monitored.

To create access keys, click the user icon (), then click Access Keys.

When integrated with MLRun 8.0 and higher, in certain wizards, you must add an access key that includes the control plane, for example the create and deploy functions, and the create job and scheduled job. The Auto-generate access key, in the Resources section of the wizard, is selected by default in these wizards. Alternately, you can clear the checkbox and add a custom access key.

Identity Provider (IdP) The authentication of the user credentials can be done locally, using the platform's built-in user management, or using an external Identity Provider (IdP)—currently Microsoft Active Directory (AD). When an IdP is configured, it is used to authenticate the identity of all its imported users in the platform. This doesn't prevent you from also defining local users and using the platform to authenticate them. For more information about using an IdP, see Using an External Identity Provider (IdP).

Authorization

Authorization is the process of granting a user permission to perform a specific action or access a specific resource based on predefined authorization rules. To support authorization, the platform uses policies and permissions that govern the ability to access resources.

• Management policies are assigned to users and user groups to determine management-related permissions. For example, the permission to create a storage pool or restart a cluster is reserved to users who have the IT Admin management policy, and the permission to access the data is reserved to users who have the Data management policy.
• Project membership is assigned to users and user groups to control access to projects and the levels of access. Project membership does not include access to project data.
• Data-access policies define fine-grained rules for determining data-access permissions. These policies are used as part of a multi-layered data-access authorization scheme, which also involves the Data management policy and POSIX ACLs.

Management Policies

Every user and user group (whether locally created or imported) must be assigned one or more of the predefined management policies. These policies define resource-access permissions with management aspects that are applicable globally throughout the platform. The management policies are assigned by a security administrator, which is any user with the Security Admin management policy, including the predefined security_admin user. For more information about user management in the platform, see Platform Users.

Predefined Management Policies

These are the predefined management policies that a security administrator can assign to users and user groups:

• Application Admin—full rights to all container operations, such as creating data containers, and defining data-access policies; view and use application services for the current user; view the pipelines dashboard.
  All locally created and imported users in the platform (but not the predefined users) are automatically assigned this policy.

• Application Read Only—view all reports without editing; view and use application services for the current user; view the pipelines dashboard.

• Data—access data and run application services. The specific access level is derived from the data-access policies and POSIX ACLs.
  This policy allows the implicit creation of data sessions, which are used for securing access to data.
  All locally created and imported users in the platform (but not the predefined users) are automatically assigned this policy.

• Developer—the only policy whose users can create projects, and manage and develop Nuclio serverless functions. (Developers can only see the projects that they are members of.) All locally created and imported users in the platform (but not the predefined users) are automatically assigned this policy.

• IT Admin—full rights for all IT operations, such as defining storage pools, stopping and starting a cluster, and changing the cluster state from offline to online. This policy includes permissions for viewing event logs and for managing cluster support logs; for more information, see Logging, Monitoring, and Debugging.
  The predefined tenancy_admin user is assigned this policy together with the Tenant Admin policy.

• Project Security Admin—view all projects, perform all admin tasks on the project, and change a project owner to any user. This policy does not give rights to modify the entities underneath such as features, jobs etc., and does not give rights to delete a project.

• Project Read Only—view all projects. A user with the Project Read Only management policy that is not a member of the project cannot drill down to see the project objects such as function, jobs, and so on.

• Security Admin—full rights for managing users and user groups. This includes creating and deleting users and user groups, assigning management policies, and integrating the platform with a supported identity provider (see Using an External Identity Provider (IdP)). (All users can view information for their own user profile and edit relevant properties, such as the password, email address, and first and last names.) This policy also includes permissions for viewing audit event logs. For more information, see Logging, Monitoring, and Debugging.
  The predefined security_admin user is assigned this policy.

• Service Admin—full rights for managing application services, including creating, configuring, restarting, and deleting user-defined services, configuring and restarting relevant default services, and managing service logs. This policy also includes permissions for viewing the pipelines dashboard and for viewing application-service logs from the log-forwarder service. For more information, see Logging, Monitoring, and Debugging.

• Tenant Admin—full rights for managing tenants, including creating and deleting tenants.
  The predefined tenancy_admin user is assigned this policy together with the IT Admin policy.

Project members

Project membership provide segregation between the various projects and their users. You can specify project members (individual users or user groups) for each project. Each project members has a role that controls access to projects management and the levels of access. It ensures that only users that are members of a given project can view and manage the project.

To access the Projects page to view and manage projects, users must have one of the following management policies: Developer, Project Security Admin, Project Read Only.

An individual user's permissions (or a user group's permissions) for any one project are the sum of the management policy and the project membership role permissions for that user or group, for that project. Project membership can be managed by users with the Project Admin policy, and by the project's owner.

Project members have one of the following roles:

• Admin—View, and add and remove members, and change their member role, and all the privileges of the Editor role.
• Editor—Edit the project. The editor has management and configuration rights for the project, for example changing function code, and creating a job. It cannot modify members or change ownership.
• Viewer—View all project content, but cannot create, edit, or delete objects in the project.

These roles are control plane roles only. Project data is managed on a different layer. Data permissions for data that resides in Iguazio’s data layer are controlled by the data access policies and POSIX ACLs.

The user that creates the project is the owner and is also an admin member of the project. The owner has management and configuration rights for the project, for example: changing function code; creating a job; and adding and modifying admin, editor, and viewer members. The owner can change the ownership to another user. In this scenario the original owner remains an admin member.

To help understand the permissions, here are a few examples:

• User x has Project Admin management policy and is a Viewer member of project p. The viewer member gives read permissions and the Project Admin policy gives permission to update the owner of project p.
• User y has Developer management policy and is an Admin member of project p. The Developer policy gives permission to create projects, and the Admin member allows the user to add and remove members, and change their member role for project p.
• User z has Developer management policy and is a Viewer member on a project p. But user z is also part of a group that has an Editor role on that project. User z effectively has the Editor role on project p and can create projects with the Developer policy.

The project owner and the number of members are both displayed in the left pane of the individual project pages. Click Change or Manage to modify the owner, or members and their permissions.

Upgrading from pre-v3.2.0

During the upgrade you must designate one user to own all of the migrated projects.

After upgrade to v3.2.0, all pre-existing projects are associated with a (dynamically updated) group of all users, named all_users, such that all users have access to all projects (this is the pre-upgrade behavior). After the upgrade you can assign members to each project.

Data-Access Authorization

The platform uses a multi-layered data-authorization scheme: each data-service operation—read, write, update, delete, etc.—is processed and examined in three layers, to ensure that the environment is protected and secured. Each layer can add to the restrictions of the previous layer:

The "Data" Management Policy

As a preliminary step to accessing data in the platform, a user must have the Data management policy. This policy enables the implicit creation of data sessions, which are used for securing access to data. (The tenancy_admin and security_admin predefined users do not have this policy and therefore cannot access data or view the dashboard's Data page.)

POSIX ACLs

Use the portable operating-system interface access control lists (POSIX ACLs) to define file-system permissions.

Data-Access Policies

Use the Data-access policies to define fine-grained policies for restricting access to determine whether to grant or restrict access to a specific data resource and to what extent. For example, to create a subnetwork (subnet) whitelist, define interface data-access eligibility, restrict the right to read a payments table that contains sensitive data to members of your organization's finance group (restrict access to a table only to specific user groups), limit the write privileges for updating the online transactions stream to members of the operational team (give some users only read-only permissions for a specific file). See additional information in the Data-Access Policy Rules section.

The following diagram illustrates the platform's multi-layered data-authorization scheme:

In most solutions, too many policy rules that need to inspect every data operation will come at a cost and may cause a performance degradation and low throughput, as the inspection takes time. However, in the MLOps Platform, the data-access policy rules are compiled and stored in an optimized binary format on every policy change—rule addition, removal, or update. This allows the platform to process the rules in a fast and effective manner, resulting in high-performance processing for each data request, in line rate, while keeping the environment highly secured.

POSIX ACLs

When you create a project, a directory is created in the projects container for that project and it serves as the default path for project files. This directory is created by the user who is the owner of the project, and by default is accessible to users that belong to the same primary POSIX group as the owner. When other users attempt to access the project files, they need to belong to the same group as the project owner. To restrict access to specific users:

1. Add specific users and/or groups to directories and files as relevant.
2. Deny access to all users and groups.

Users must have the Data management policy, and they must have access to any other locations where related data is stored.

Data-Access Policy Rules

Users with the Application Admin management policy (such as the predefined security_admin user) can define a set of fine-grained data-access policy rules. These rules are checked for each data operation and are used to determine whether to allow or deny the data request. Data-access policy rules are defined in the context of a specific data container and apply to all data objects in the container, regardless of their type.

• Defining Rules
• Rules Processing
• Predefined Rules and Layers
• Examples
• Data Categories

Defining Rules

Data-access policy rules are managed from the Data | <container> | Data-Access Policy dashboard tab, which displays the predefined data-access policy rules and layers and options for adding and editing rules, groups, and layers, as demonstrated for the "users" data container in the following image:

A rule must belong to a data-access layer. You can either add rules to one of the predefined layers for the parent data container or create your own layer: select the New Layer option from the New Rule drop-down menu in the top action toolbar; in the new-layer dialog window, enter the layer name and select Create. The following image demonstrates creation of a new layer named "Default layer":

You can add rules directly to a layer or group multiple rules into one or more rule groups within a layer. To add a new group, select the New Group option from the New Rule drop-down menu in the action toolbar; in the new-group dialog window, enter the group name, select a parent layer, and select Create.

The purpose of the layers and groups is to help you manage your rules and easily reorder rules to change the processing logic, as explained in the Rules Processing section. You can rename a layer or group by selecting and editing the name in the rules table, and you can delete it by selecting the delete icon () for the relevant table entry.

To add a new data-access policy rule, select the New Rule option from the action toolbar; in the new-rule dialog window, enter the rule name, select a parent layer and optionally a parent group, and select Create. The following image demonstrates creation of a new rule named "Rule 1" in a layer named "Default layer":

After you create a rule, select it from the rules table to display the rule pane and define the permissions for accessing the data based on one or more of the following characteristics (match criteria).

Sources

A rule can be restricted to specific sources.

Currently, the platform support an Interfaces source type, which is an interface for accessing the data:

• Web APIs—the platform's web-APIs, which are available via the web-APIs service (webapi)
  • V3io Daemon—the platform's core daemon service (v3io-daemon), which connects application services (such as Spark and Trino) to the platform's data layer.
  • File system—Linux file-system operations.

Users

A rule can be restricted to a specific list of predefined users or user groups, excluding the predefined group all_users. Note that user-group match criteria in data-access policy rules are applicable only to the primary group of the user who attempts to access the data.

Resources

A rule can be restricted to specific data resources.

A resource can be defined as a path within the container, such as the path to a table or stream or to a subdirectory or file.

A resource can also be defined as a logical category of data—such as audio, video, logs, or documents. For a list of all resource data categories and the file extensions that they represent, see Data Categories.

After defining the match criteria for the rule, you define the data-access permissions to be applied when there's a full match. You can select whether to allow or deny access to the data and to what extent. For example, you can grant only read permissions, deny only the create and delete permissions, or allow or deny full access. The following image demonstrates full data-access permissions:

Rules Processing

The rules are processed for each data operation according to the order in which they appear in the dashboard. You can change the processing order, at any time, by changing the order of the data-access policy rules in the dashboard: you can change the order of the rules and rule groups within each container data-access layer; change the order of rules within each group; and change the order of the layers.

When a full match between the operation and a policy rule is found, the processing stops and the data accessibility is set according to the permissions of the first-matched rule. A match is identified by checking all components of the rule. All match-criteria rule sections are accumulative ("AND") but the values in each section are alternative ("OR"), except where otherwise specified. See the examples for a better understanding.

Predefined Rules and Layers

The platform predefines the following data-access policy layers and rules for each data container, except where otherwise specified:

• System layer—A system-administration layer that has the following predefined rule:

  • Backup—This rules grants the predefined "sys" backup user full data access, to support data backups. It's recommended that you keep this rule as the first rule in your processing order.

    

• Monitoring layer—A monitoring-service layer that has the following predefined rules:

  • Monitoring—This rule is defined only for the predefined "users" container and grants the predefined "monitoring" user full data access to the monitoring directory, which is automatically created in the root directory of this container for use by the monitoring service.

    

  • No access—This rule denies the predefined "monitoring" user all data access. Note that on the "users" container, this rule must not precede the "Monitoring" rule, as the first rule takes precedence (see the rules processing order).

    

Examples

The predefined data-access policy rules provide examples of granting and restricting data access for a specific user and/or resource (data directory). Following is a step-by-step example of adding your own custom data-access policy rules from the dashboard Data-Access Policy tab.

1. Create a new "Default layer" layer: from the top action toolbar, select the drop-down arrow on the New Rule button and select the New Layer option from the menu. In the Create new layer dialog window, enter your selected layer name—"Default layer" for this example:

   

   Keep the new layer after the predefined layers in the rules table (default).

2. Define a custom "IT Logs" rule that grants members of the "it-admins" user group full permissions to access any log or document file in either the system/logs or it directories in the parent container:

   Note

   To define and test this rule, you need to create an "it-admins" group from the Identity | Groups dashboard tab, assign users to this group, and create the directories that are specified in the match criteria. Alternatively, you can change the match criteria to accommodate your environment and needs.

   1. From the top action toolbar, select the New Rule option. In the Create new rule dialog window, enter your selected rule name—"IT Logs" for this example.

      

   2. Select the Users/Groups cell of the "IT Logs" rule in the rules table to display the Users tab in the rule pane on the right. In the Users/Groups input box, start typing "it-admins" and select this group from the list.

      

   3. Select the Resources tab in the "IT Logs" rule pane. In the Paths section, select the plus sign (), enter /system/logs in the input box, and select Apply. Repeat this step but this time enter the path /it.

      

   4. In the Permissions tab, keep the default allow-all permissions.

3. Define a custom "Deny All" rule that denies all data access, as recommended in the rule-processing section:

   1. Create a new rule in the "Default layer" layer and name it "Deny All".

   2. Select the Permissions cell of the "Deny All" rule in the rules table. In the Permissions rule tab, select the Deny option from the permissions drop-down box and keep all permission check boxes checked to deny all data-access permissions.

      

      Note

      • The deny-all rule must be the last rule in the data-access policy rules table; any rules that appear after it will be ignored. You can move the rule to another layer, if you wish.
      • You might want to disable this rule during the initial stages of your development and testing, as it blocks all data access that isn't explicitly permitted in other (preceding) data-access policy rules.

4. Select Apply Changes from the pending-changes toolbar to save your changes:

   

You can now see your new layer and rules in the data-access policy rules table:

Data Categories

The following table lists the supported data categories, which can be used to define a resource for a data-access policy rule, and the file extensions that each category represents:

See Also

• Platform Users
• Securing Your Web-API Requests
• Sessions Management API [Beta]
• Working with Services
• Security software specifications and restrictions