Lesson
2.3 Creating workspaces
Workspaces in Mage Pro provide isolated environments for developing and managing data pipelines. Each workspace operates as a separate container instance with independent resource allocation and configuration management.
Understanding workspaces
Think of a workspace as your personal data engineering environment. Each workspace includes a containerized Python environment, persistent storage for your code and data, Kubernetes-based resource allocation, and role-based access control for team collaboration.
This isolation is crucial in professional data engineering - you can work on customer analytics while your colleague develops marketing pipelines without interfering with each other's work.
Prerequisites
To utilize workspaces, your organization needs this feature enabled by contacting Mage for an account upgrade. The Mage team will enable this feature for your organization.
Creating your first workspace
Step 1: Access the workspace interface After logging into Mage Pro and opening your cluster, a new tab will open to your workspaces interface. Click the "Create new workspace" button.
Step 2: Configure basic settings Give your workspace a descriptive name that indicates its purpose and environment (e.g., "customer-analytics-dev" or "marketing-pipelines-test"). This naming pattern prevents confusion when working across multiple environments.
Step 3: Set up resource allocation (optional) Configure your workspace's computing resources to balance performance and cost. For learning purposes, modest resources work fine. As you work on larger projects, you'll develop intuition for what your workflows require.
Step 4: Configure environment variables (optional) Set environment variables that tell your workspace how to connect to databases and find files. The most important is USER_CODE_PATH, which tells your workspace where to find your project files. Most beginners won't need to change these settings.
Step 5: Set lifecycle policies (optional) Configure auto-termination to automatically shut down your workspace after inactivity (e.g., 1800 seconds for 30 minutes). This prevents running cloud resources indefinitely and controls costs.
Step 6: Deploy and access Click "Create" and wait a few minutes for deployment. You'll see status indicators showing progress. When it shows "Running", click the "Open" button to enter your workspace.
Advanced configuration options
Sharing secrets across workspaces: Create an environment variable SHARE_SECRETS_ACROSS_WORKSPACES set to 1 to share secrets across workspaces within a cluster.
Environment variable inheritance: Set INHERIT_ENV_VARS=1 to inherit environment variables from the main cluster. Use USER_DEFINED_VARS to control which variables are inherited:
Managing workspace users
Global roles: Project administrators assign global roles that apply across all workspaces through the Users tab in the main project interface.
Workspace-specific roles: Individual workspace owners can assign user roles, modify assignments, and remove user access specific to their workspace.
Updating workspace configurations
When environment variables change in your base cluster, existing workspaces require manual updates:
Click the workspace you want to update
Click the "Update" button
Toggle "update workspace settings" to true in the Kubernetes configuration
Click "Update"
The container takes several minutes to restart with new configurations.
Conclusion
Creating workspaces is a foundational skill in Mage Pro that provides isolated environments for your data engineering projects. Each workspace becomes your personal development laboratory where you can experiment and build without affecting other projects or team members.