The Nuclio Serverless-Functions Service

On This Page

Overview

Iguazio's Nuclio Enterprise Edition serverless-functions framework — a leading open-source project for converging and simplifying data management — is integrated into the platform. Nuclio is a high-performance low-latency framework that supports execution over CPUs or GPUs and a wide array of tools and event triggers, providing users with a complete cloud experience of data services, ML, AI, and serverless functionality — all delivered in a single integrated and self-managed offering at the edge, on-premises ("on-prem"), or in a hosted cloud.

You can use Nuclio functions, for example, to:

  • Collect and ingest data into the platform and consume (query) the data on an ongoing basis. Nuclio offers built-in function templates for collecting data from common sources, such as Apache Kafka streams or databases, including examples of data enrichment and data-pipeline processing.

  • Run machine-learning models in the serving layer, supporting high throughput on demand and elastic resource allocation.

Nuclio can be easily integrated with Jupyter Notebook, enabling users to develop their entire code (model, feature vector, and application) in Jupyter Notebook and use a single command to deploy the code as a serverless function that runs in the serving layer. For examples, see the platform's tutorial Jupyter notebooks.

Nuclio is provided as a default (pre-deployed) shared single-instance tenant-wide platform service (nuclio). Users can disable, enable, and restart this service, as well as configure the Docker Registry for storing the Nuclio function images. See also the Nuclio restrictions in the Software Specifications and Restrictions documentation.

Configuring the Nuclio Service

Pod Priority for User Jobs

Pods (services, or jobs created by those services) can have priorities, which indicate the relative importance of one pod to the other pods on the node. The priority is used for scheduling: a lower priority pod can be evicted to allow scheduling of a higher priority pod. Pod priority is relevant for all pods created by the service.
Eviction uses these values to determine what to evict with conjunction to the pods priority. See more details in Interactions between Pod priority and quality of service.

Pod priority is specified through Priority classes, which map to a priority value. The priority values are: High, Medium, Low. The default is Medium.

Configure the default User functions default priority for a service, which is applied to the service itself or to all subsequently created user-jobs in the service's Common Parameters tab, User jobs defaults section, Priority class drop-down list.

The priority applies to the pods of the Nuclio-created functions.

Resources for User Jobs

When you create a pod in a Nuclio function, the pod has default CPU and memory limits. When the job runs, it can consume resources up to the limits defined. The default limits are set at the service level. You can change the default limit for the service, and also overwrite the default when creating a function.
See more about Resource Management for Pods and Containers.

When creating a service, set the Memory and CPU in the Common Parameters tab, under User jobs defaults. When creating a function, overwrite the default Memory, CPU, or GPU in the Configuration tab, under Resources.

Node Selection

You can assign jobs and functions to a specific node or a node group, to manage your resources, and to differentiate between processes and their respective nodes. A typical example is a workflow that you want to only run on dedicated servers.

When specified, the service or the pods of a function can only run on nodes whose labels match the node selector entries configured for the service. You can also specify labels that were assigned to app nodes by an iguazio IT Admin user. See Setting Labels on App Nodes.

Configure the key-value node selector pairs in the Custom Parameters tab of the service.

If node selection for the service is not specified, the selection criteria defaults to the Kubernetes default behavior, and jobs run on a random node.
Node selection is relevant for all cloud services.

See more about Kubernetes nodeSelector.

You can also configure the node selection for individual Nuclio functions when creating a function in the Confguration tab, under Resources, by adding Key:Value pairs.

External System Docker Registry

Note
External docker registry is relevant if the platform is running with MLRun v1.0.5 or higher.

By default, the Nuclio service on the default tenant is configured to work with a predefined, tenant-wide Docker Registry service, which uses a pre-deployed, local, on-cluster Docker Registry. You can change the configuration of the Nuclio service to work with an off-cluster Docker Registry service.

The system registry ensures that all container images required by the system to operate are pulled from the given address. This is supported for managed Kubernetes integrated with cloud-provided container registry. (e.g.: AKS with ACR, EKS with ECR). The URL is the container registry address. When deploying multiple systems to the same container registry, you can use different URLs, for example xyz.my-ecr.amazon.com/some-unique-name. This is recommended to avoid overriding the existing container images.

Either select an external Docker Registry from the drop-down list, or press Create new..

Parameters:

  • URL: Required.
  • Username and password: Optional
  • Image prefix. Optional. When defined, the image prefix is appended to the container images (that were built in MLRun) when they are pushed by an Iguazio service (e.g. MLRun) to the registry.

The external registry does not support explicit authentication. You must ensure that the k8s is deployed with a role that allows it to read/write to that given registry.

Tip
If you're using the ECR/ACR for both the user custom registry and the external system docker registry, you can distinguish between the registries with suffixes. For example:
my-ecr-address.ecr.com/my-igz-system-runtime for the custom system container registry
my-ecr-address.ecr.com/my-igz-system for the custom user registry

When creating a registry on ECR

  • If the permissions for the ECR are already set as part of the cluster deployment (using the EC2 IAM policy), then use ecm.com as the URL and leave the username and password blank. (EC2 instances are attached with roles allowing it to work with the ECR.)
  • If the ECR was not used for the cluster installation:
    • URL: The ECR URL (in the format <aws_account_id>.dkr.ecr..amazonaws.com).
    • Username: AWS access key ID
    • Password: AWS secret access key
Note
When using ECR as the external container registry, make sure that the project secrets AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY have read/write access to ECR.

The access keys or the EC2 IAM policy must have these permissions:

{ 
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ecr:CreateRepository",
                "ecr:GetAuthorizationToken",
                "ecr:BatchCheckLayerAvailability",
                "ecr:BatchGetImage",
                "ecr:CompleteLayerUpload",
                "ecr:GetDownloadUrlForLayer",
                "ecr:InitiateLayerUpload",
                "ecr:PutImage",
                "ecr:UploadLayerPart"
            ],
            "Resource": "*"
        }
    ]
}

See more details in EKS and AWS Vanilla Kubernetes.

See Also