The Right Way to Scale Access Control Implementing ABAC & Format Preserving Encryption (FPE)
Joint Blog
Written by: Mike Mitrowski, Snowflake Global Field CTO, and Noam Dror, SecuPi VP Solution Engineering
Large Snowflake deployments often involve thousands of users and roles. Access is managed by assigning complex combinations of roles to users and policies which increases exponentially the number of roles (referred to as “role sprawl”), while affecting maintenance cost and time required to provision access on a “need to know” basis.
Role management sometime leads to copy of role with one of few changes which at the end leads to more roles than users.
Any time where there is a re-org or new data set a role redesign project needs to be initiated.
In this blog, we will be describing the root cause of role sprawl and management, while ensuring ABAC scalability.
Main causes for Role Sprawl and how to effectively evolve RBAC into ABAC
The increase in the number of analysts and data scientists accessing sensitive data sets with tighter access controls imposes exponential increase in the number of roles. Main causes for going into a role sprawl include:
- Hierarchical attributes and tags
Implementing location attributes (e.g., policies based on location hierarchy such as both European and Swiss locations – each with different controls), product classification tree, customer categorization hierarchy, organizational charts and clearance levels (e.g., C1-C4) are all hierarchical attributes that cause exponential increase in complexity when enforcing fine-grained access controls. Managing hierarchies in roles is a management headache. - Row-level filtering based on multiple attributes
Filtering rows based on several conditions taking into consideration user and customer attributes is complex when imposed across hundreds of tables and data sets.
This becomes even more challenging when the filtering attributes are not replicated in each table as policies require the filtering attributes present in the table (e.g. reference tables)
- Dynamic attribute enforcement
Adding dynamic attributes into access restriction is required for addressing end-user current location, validating up-to-date desktop security health-check, risk score/continuous authentication, purpose or customer’s consent. Solving at scale dynamic pulling of attributes, synchronization and enforcement (push-down) is complex when trying to do it directly at Snowflake. A better approach is required to authorize based on real-time attributes.
SecuPi has been a Snowflake partner for the last four years, serving dozens of Fortune 500 customers.
SecuPi’s platform includes a Centralized Management Server to dynamically gather attributes from all sources and enforce policies natively to Snowflake natively (by defining policies and pushing attributes) or through distributed gateways/agents (Enforcers).
Providing several policy Enforcer options ensures ABAC scale, security and compliance.
In order to augment existing policies and views, SecuPi provides a centralized User Attribute Repository (UAR) that pulls, synchronizes and organizes attributes per user. This user attribute repository can be updated in real time with a REST API call and a standard SCIM 2.0 API call. At the same time, there is integration with SSO/Federation systems to receive real time attributed when users login to an application of a database. SecuPi enforcer (inline) pull these attributes based on the end-user identity, location and state.
It enforces ABAC and client-side encryption on any access to protected data – BI tools, data applications, direct DB tools and Snowflake UI and APIs.
SecuPi self-contained Enforcers can both set policies, external functions and dynamic masking in Snowflake by automatically rewriting the query by adding appropriate constraints or by filtering the result-set returned from the data source including any filtering, decryption, encryption or masking of sensitive data (e.g., filtering out VIP customers or masking patient PHI data based on their state). ensure user queries conform to the Users entitlements and data attributes as defined by the policies.
It uses all three options to ensure access to Snowflake conforms with ABAC policies.
SecuPi’s simple yet comprehensive policy user interface enables ‘point and click’ customizable data access rules using plain language and logical operators. These data access rules incorporate attribute variables, so as attribute values vary, the data access decisions adjust dynamically and automatically.
SecuPi Centralized Management Server can be deployed on-prem or in customer’s cloud using a highly available Kubernetes cluster. It pulls context and attributes from multiple sources, including LDAP, AD, Data Catalogs (e.g. Collibra, Alation, Glue), HR applications and Identity sources, such as Okta and SailPoint. SecuPi enforcers then pulls only the relevant attributes which are cached in-memory and synched . This ensures that the Enforcers have the required attributes in the transparent data-gateway memory, without having the performance overhead of polling each attribute source per request.
The SecuPi Management Server polling rate of attributes can be configured and synchronized. SecuPi comes with preconfigured integrations that use either REST API, Web Service, Import of JSON/XML, Polling attributes from SQL and NoSQL stores and COTS applications to name a few.
With increasing amounts of sensitive data lifted from operational systems, organizations need to adopt a Zero Trust concept of “Security by Design.”
This can be achieved by encrypting sensitive data elements at the source—as data leaves its operational sources (e.g., on-prem Oracle or Salesforce), while decryption (back to clear text) is performed on the client-side based on a “need-to-know” basis.
SecuPi is a “defense in depth” control. Since it can apply access control in snowflake but also outside of snowflake so control cannot be bypassed by privileged users of snowflake and provide a parallel audit log that can be compared to snowflake log to see if anyone if changing or erasing the native log.
SecuPi ABAC and client-side encryption for Iceberg
With Iceberg fast adoption, accessible through various compute engines as well as directly on cloud storage (e.g. S3, Azure Blob Store), encrypting critical columns at-source while exclusively decrypting them for authorized users only, enforcing fine-grained access controls ensures Zero Trust.
SecuPi has built agents for all common ETL and ELT tools – including Kafka, Confluent, Databricks, Spark, Glue, Azure Data Factory, Talend, DataStage, Informtica and Attunity.
The agents are downloaded from the SecuPi management server and configured on the ETL/ELT servers. By configuring the data-source sensitive columns or the columns within a topic – SecuPi automatically encrypts, masks or filters the sensitive data at-source.
Once the encrypted data lands in Iceberg – the SecuPi Enforcers for Snowflake will apply ABAC to determine if conditions are met to re-identify/decrypt the sensitive data queried from Iceberg. This ensures that all ways to sensitive data in Iceberg are monitored and secured by design.
A common use case for encryption is to address customer data sovereignty requirements as the decryption of sensitive data is performed using a transparent data-gateway (“Enforcer”) that is installed in-country VPC. Users accessing Snowflake in-country connect through the Enforcer. The data decryption keys remain in-country and they are never shared with Snowflake – referred as “client-side encryption” with “Hold Your Own Keys” (HYOK) – as the data decryption keys remain in-country and they are never shared with Snowflake.