The Security Boundary Includes the Workaround

We spent years building controls around the transfer of sensitive customer data. Our customers often preferred FTP.

That sentence captures a problem that security policies tend to obscure: the approved system is not the whole system. If customers and employees find the approved route difficult enough, the real workflow expands to include every exception, shortcut, and improvised transfer they use to get the work done.

We did not support installed software. We operated as an application service provider. But our business moved large amounts of customer data between employers, health care companies, insurers, payroll companies, and our own systems. The data was consequential, private, and eventually subject to regulatory obligations.

We built data-interchange infrastructure to handle it. Many customers hated that infrastructure.

The protocol debate was not really about protocols

Every new interchange seemed to begin with a discussion about method. FTP, SFTP, PGP, GPG, SCP, HTTPS, AS2, secure email: which technology would we use, and whose infrastructure would receive the connection?

“Theirs or ours?” was a recurring argument.

We were sometimes uncomfortable with customer-operated endpoints. Customers scrutinized our controls and later examined our SOC 2 reports, yet sometimes asked us to send sensitive data through infrastructure we could not evaluate or govern. From their perspective, our preferred mechanism introduced unfamiliar software, enrollment, keys, passwords, rotation schedules, and operational dependencies. Their existing FTP process was understood and already worked.

Neither side was behaving irrationally. Each was optimizing for a different boundary. We were responsible for controls inside ours. Customers were responsible for completing business processes inside theirs. The transfer sat awkwardly between the two.

For a time, we permitted conventional FTP but required the files themselves to be encrypted. On paper, that separated the security of the data from the security of the transport. In practice, it required people in two organizations to carry out an unfamiliar additional step correctly on every exchange.

Customers sometimes did not encrypt the files. Our employees sometimes failed to catch it. Key distribution and rotation created their own support burden. When something failed, the pressure was to make the file move...not to admire the control design.

The encryption was sound when it was used. The workflow was not sound because using it remained a human choice.

When the risk stopped being theoretical

In the early 2000s, an internet-facing FTP server we operated was misconfigured. Some customer files on it had not been encrypted, and customer data was exposed.

That was our failure. We had built a control that still depended on customers and employees performing an inconvenient extra step correctly every time. Eventually, someone did not.

I do not mention the incident because it was novel. It was not. I mention it because the failure did not require an exotic cryptographic attack or an ingenious defeat of a mature security architecture. The attacker reached a familiar transfer service, and some of the files there lacked the protection our policy required.

The policy described the system we intended to operate. The exposed files revealed the system we actually operated.

The custody tax

Our business and controls matured greatly over time. The later environment included per-customer segregation, encryption in transit and at rest, active scanning of incoming files, access restricted to customer-assigned teams, formal access requests, retention rules, and backup controls. Evidence that these controls operated was sampled and tested during SOC 2 audits.

Those controls mattered. They also illustrate what I think of as the custody tax.

Once another organization’s data enters yours, protecting the network connection is only the beginning. You must decide where the data can land, who can see it, how it is segregated, which systems can copy it, how long it remains, whether it enters backups, how deletion works, and how you will prove that the answers match policy.

The transfer may take seconds. The resulting obligations can persist for years.

Intermediate copies were especially difficult. A file might arrive in a controlled location and then be downloaded to an employee workstation, placed temporarily in another directory, attached to an internal message, or reproduced for an escalation. Ordinary email, and later, chat, created new channels through which well-intentioned people could move sensitive information around the controls built for formal interchange.

We repeatedly reminded customers not to send sensitive employee information through unencrypted email. We repeatedly reminded our own people to use the approved mechanisms. Necessary as those reminders were, they were evidence that the easier path and the safer path were still different paths.

A private support portal is better—but it is still custody

The same lesson applies to remote product support.

In the first field note, I looked at customers posting diagnostic output in public forums. Moving that exchange into a private support portal plainly reduces the immediate exposure. It is a responsible improvement.

But the portal does not make the underlying questions disappear. It transfers the diagnostic archive into the vendor’s custody.

What happens next?

• Which support personnel can retrieve it?
• Can an engineer download it to a workstation?
• Is it copied into an engineering issue or an internal conversation?
• Can support-platform employees or other subprocessors access it?
• Is deletion tied to ticket closure?
• Does the file remain in backups afterward?
• Can the customer see who accessed it or compel its deletion?

The archive may contain far more than the immediate problem requires. Red Hat, for example, warns that an sos report may contain data considered sensitive and recommends reviewing its contents. That is responsible advice. It also acknowledges that a general-purpose collection tool cannot know everything a particular customer considers sensitive.

The customer is again asked to inspect an unfamiliar artifact while trying to restore a failed system.

When the support repository becomes the target

In October 2023, Okta disclosed unauthorized access to its customer support system. Okta support sometimes asked customers to upload HTTP Archive, or HAR, files to reproduce browser problems. Those files can include cookies and session tokens. The intruder accessed files uploaded in support cases, and exposed session tokens were used in attempts to compromise customer environments.

Cloudflare’s account of the incident describes tracing malicious activity in its environment back to a token compromised through Okta’s support system.

This does not demonstrate that private support portals are inherently unsafe. It demonstrates that repositories of customer diagnostic material become part of the security boundary—and potentially valuable targets in their own right.

“Private” answers the question of public visibility. It does not answer the questions of collection, access, retention, propagation, or consequence.

Reduce custody, not only exposure

Looking back at our data-interchange operation, I can imagine a different model.

Our system could have requested temporary access to a named data service at the customer site. The customer could approve the request directly or through an agreed policy. The intended receiving application could then read the required data through that connection without an employee staging another copy on an FTP server (maybe in a DMZ...another copy), attaching it to email, or opening either network to unsolicited inbound traffic.

The data would still cross an organizational boundary. The receiving system might still process or retain it for a legitimate business purpose. Application authorization, data segregation, retention, and audit obligations would still matter.

What could disappear are many of the intermediate exchange copies, shared transfer credentials, exposed listeners, and manual handling steps that made the old workflow so difficult to govern.

The same principle applies to product support. Sometimes a diagnostic artifact truly must be captured and transferred. But extraction should be a deliberate exception with visible custody, not the universal first step in diagnosis.

A secure channel is necessary. It is not sufficient when the surrounding work encourages people to bypass it or creates copies nobody can confidently account for.

If the approved route is hated, the security boundary includes every workaround people use to avoid it.