Polish AI harness. How to give companies capabilities and people control
Series · part 1 of 3: technology
Author: Andrzej Okoński, Head of Software Development
The value of artificial intelligence in a company depends on the knowledge it can access, the tools at its disposal and the boundaries within which it may act. A well-designed harness brings these elements together into an environment that both the organisation and the employee keep under control.
Imagine a salesperson asking an AI assistant: “Prepare an offer for the client based on our last conversation, the current price list and the terms of cooperation”. It is a hypothetical example, but it shows well the questions worth asking before deployment.
Where will the assistant get the price list from? Will it tell the current version from an archived one? Will it see another team’s confidential arrangements? Where will the content of the conversation with the client end up? And finally: will it prepare a draft, or will it also send it?
Each answer means a specific design decision. These decisions determine whether AI becomes a useful work tool and whether the employee understands what they are responsible for.
What we mean by an AI harness
An AI model is like a horse: it has strength, but it is the harness that decides where it will run. That is why the same model can produce completely different results in two companies. Our SAIE ecosystem is such a harness for any model: the organisation can change the model, while quality and control remain.
In this series, we call a harness the layer of technical solutions and organisational rules that connects an AI model with the real work of a company. It covers access to knowledge, the choice of models, integrations with systems, permissions, oversight, event logging and cost control.
It is the harness that defines what information the model receives, what actions it may propose or perform and at what point the decision belongs to a human. It should also make work easier: suggest the right source, explain limitations and point out what is missing to complete a task.
“Polish” means here a fit with the language, documents and ways of working of Polish organisations, taking into account European rules on data protection and accountability. It also means the ability to make a conscious choice of vendor, model and place where information is processed.
Such a fit is worth checking on real materials: a Polish form, industry abbreviations, job titles, and dates and amounts written in the local format. Does the assistant distinguish the roles in the company’s procedure? Does the employee understand an error message and know whom to turn to for help? These are questions worth asking every vendor, including us.
In this sense, sovereignty is a practical ability to decide: what technology we use, what we entrust to it and how we can change the vendor. It requires checking the entire data path, including external models, integrations and diagnostic records.
1. Deliver the right knowledge
In our example, the salesperson needs an offer that complies with the company’s current rules. The harness should therefore give the assistant the current price list, the right terms of cooperation and the arrangements made with this client.
This kind of work is supported by RAG: finding the relevant fragments of documents and passing them to the model as the context of its answer. In our TWIN:DESK, every answer based on documents comes with a quotation from a specific fragment, and the assistant recognises document versions and status, so it tells the current price list from an archived one. The quotation gives the user a point of reference when checking the answer.
When designing such an environment, it is worth requiring information on the document version, its period of validity and the owner responsible for keeping it up to date. Access to materials should take the employee’s permissions into account already at the search stage.
Let us assume that an approved rate for an additional service is missing. The expected behaviour is to point out this gap and refer the matter to the person responsible for pricing. Generating a complete offer at any cost would be a bad criterion of success. In TWIN:DESK, this is what the Knowledge Gap Register is for: the assistant reports the gap instead of guessing.
The source makes checking easier, and the final word belongs to a human. That is why, in our scenario, the employee opens the relevant fragment of the price list and compares it with the assistant’s proposal.
2. Control the data path
A request to prepare an offer may contain contact details, the history of negotiations and information about the margin. The organisation should decide in advance which of these may reach the chosen model, for what purpose and to what extent.
It is worth starting by assigning confidentiality classes to documents and linking them to the rules of access to models. For some tasks a cloud service will be acceptable, for others a model running in an environment controlled by the organisation. In SAIE Pantera, such a model can run even on a server without internet access.
Another practical requirement is to limit the information passed on to what the task needs. To build the structure of an offer, one can use a description of the client’s needs without revealing the entire history of correspondence. Pricing will need different data, made available in an appropriately protected process.
These rules must cover emergencies. If the permitted model is unavailable, the task is suspended or passed to an approved substitute that meets the same data protection conditions.
In our SAIE ecosystem, the role of the central gateway to models is played by PROXY:AI. It selects the model according to policies and permissions, and before a query goes any further, it masks sensitive data such as a PESEL number, a NIP number or an email address. It also keeps an eye on limits and budgets, and when the permitted model is unavailable, it switches traffic to an approved backup model. We tailor these settings during deployment to the client’s specific process.
3. Match permissions to the task
Preparing a draft offer and sending it to the client have different consequences. In our scenario, the assistant may read the necessary documents and prepare a proposal. The offer is sent after an authorised person has checked the content, price, recipient and attachments. In TWIN:DESK, the agent first presents a plan for approval and only then carries it out.
These boundaries should be enforced by the applications and tools that perform the operations. An instruction written in a conversation with a model needs to be backed by real access control.
OWASP describes the risk of excessive agency: too broad a set of tools, excessive permissions and autonomy. It recommends limiting them to the needs of the task, checking permissions in the target systems and human approval before actions with significant consequences.
It is also worth testing a manipulation attempt: what will the assistant do when a document from a client contains an instruction to reveal other files? The expected test result is that access boundaries hold and the prohibited operation is stopped. The content of an analysed document should not grant new permissions.
Responsible autonomy means a clearly defined scope of independence. In our example, routine preparation of a draft can run smoothly, while the decision to send the offer remains a conscious one.
4. Make it possible to stop and explain actions
The design of a harness should also cover the moment when something goes wrong. Let us assume that the assistant repeatedly retries fetching the price list or prepares a document for the wrong client.
The employee then needs a visible status and a way to interrupt the task. The maintenance team needs information that allows it to establish at which stage the problem occurred and who is responsible for handling it.
The event log should make it possible to link a task with the model used, the configuration version, the sources and the operations performed. In SAIE, the execution history in TWIN:DESK shows every step of the agent, and PROXY:AI records model calls, costs and decision context in a log that cannot be altered. The scope of stored content, access to the records and how long they are kept require a conscious decision.
From the start, it is also worth distinguishing actions that can be reversed from those whose effects cannot be fully undone. Restoring an earlier version of a draft is a different situation from recovering information already sent to a recipient. That is why the safeguard before sending is particularly valuable in this process.
5. Measure the quality of the work done
In the case of an offer, we propose measuring the accuracy of prices, compliance with the terms, completeness of sources, preparation time and the extent of corrections made by a human. In reports created in SAIE Pantera, the figures are calculated by a calculator, not a language model, and a report with an inconsistent figure will not be published. The cost calculation should also include the work of checking and handling exceptions. The management board sees the costs of the AI itself in SAIE on a single screen, together with a forecast.
Tests should cover ordinary tasks as well as difficult situations: an outdated price list, contradictory documents, lack of access, an unavailable model or an attempt to send an unapproved version. After a change of model or configuration, it is worth rechecking a representative set of such cases.
This approach is in line with the direction set out in the NIST AI RMF Playbook: choosing measures suited to the application, defining acceptable results and monitoring performance after deployment.
For the management board, the success criterion of a pilot may therefore read: the team prepares correct offers more efficiently while keeping control over information and over the decision to send. Specific target values should be set on the basis of a measurement of the current process.
How these roles come together in SAIE
The example described helps to put in order the roles of the three elements of our ecosystem.
TWIN:DESK is the environment for everyday work. It brings together conversation with AI, documents, company knowledge and assistants. In the offer process, it is the place where the draft is prepared and the materials it is based on are checked.
PROXY:AI is the control layer for communication with models. It manages access, policies, costs and the logging of operations. In our scenario, its settings follow from the confidentiality of documents and the rules for using models adopted by the company.
CDF brings order to deployment and accountability. Our methodology leads from identifying needs through strategy, oversight, preparing people and a pilot to maintaining the solution. It helps to define the purpose of the process, its owner and the assessment criteria. CDF Platform supports the application of the methodology and the documentation of the work, and the safeguards set out according to CDF are implemented by us in TWIN:DESK, PROXY:AI and in the systems that perform the operations.
We combine these roles during deployment: we define the data flow, configure the integrations, assign responsibility and test the whole scenario together with users.
Control that helps people
The way limitations are presented matters. A message about lack of access should explain how to continue working. A request for approval should show the recipient, the content and the consequences of the operation. Missing information should lead to a specific source of help.
This is how we understand technical support for agency: the employee can understand a proposal, check it, correct it or reject it. The organisation should give them the time and the skills needed to do so.
It is also worth adopting the principle that measurements serve to assess the process and the quality of the system. The number of prompts sent to AI should not become a stand-alone measure of an employee’s value. Likewise, the possibility of talking to an assistant should complement access to support from the team.
This is the starting point for a broader conversation about wellbeing, skills development and the meaning of work.
From one process to a responsible deployment
The first step may be for the people who do a task, its business owner and IT to describe that one task together: what data is needed, what AI can do on its own, when a human decision is needed and how we will recognise a good result.
In our example, this will be an offer based on the right documents, prepared with confidentiality maintained and sent after conscious approval. On such a scenario, we show in a pilot how TWIN:DESK, PROXY:AI and a deployment run according to CDF work together.
A good Polish AI harness should increase the organisation’s capabilities and strengthen people’s ability to steer technology.
In the second part of the series, we will look at the impact of AI on employees’ agency, skills, relationships and sense of security. In the third, we will describe accountability and regulatory obligations.