> No. A few thousand other bags contained paper that had been shredded into incredibly small pieces in 1990. This material was destroyed back in 1991 after inspection by the Stasi Records Office. Hence, there is no longer any paper shredded by the Stasi at today's archive .
Shredder chads from the 90s can definitely be reconstructed. This is why the NSA has gradually decreased the maximum size permissible for chads. If they'd held on to that material they could be recovered today.
While not impossible, the volume of the destroyed document would have made this challenging even today. First, you'd have to scan them, then the software would have to be able to handle the massively increased number of combinations from having a whole bag of material rather than three pages.
Given how much of a hard time they're having with hand-torn pages, which should be a long solved problem, it's very likely that it would have been impossible in practice, in the government IT framework for a long time...
Apparently! Vol. 32 of above (per a spot check) has many docs that were shredded. My recommended method is to put papers in a bucket of liquid and then smush it. Good luck reconstructing that.
Vernor Vinge's Rainbows End includes this as a minor early plot point; libraries being fed into giant shredders that digitize and stitch together the scraps on their way to the incinerator.
I'm just wondering why the stasi didn't build a large pit somewhere in a land fill, put all the records in it and burn the lot. 16000 bags weighing what 10 kg each is something like 160 tonnes??
Burning paper in bulk for destruction in an uncontrolled process doesn't really work, you end up with huge chunks of unburnt paper because it couldn't get enough oxygen, plus the ashes may be recoverable unless mixed. Especially bad if you only get it halfway done because an angry mob shows up and stops you.
For fast emergency destruction of documents, the NSA has developed a procedure https://media.defense.gov/2021/Jul/02/2002755769/-1/-1/0/EME... where paper and oxidizer get layered in a burn barrel. "The ratio which was adopted was 65 pounds of paper for 100 pounds of nitrate" gives you an idea how much extra oxygen is required for proper combustion.
iirc it works better if you burn the paper in an agitator, like a big wire cage that turns and tumbles the mass of paper over and over until it’s all been stirred and aerated to ashes
>why the stasi didn't build a large pit somewhere in a land fill, put all the records in it and burn the lot.
In hindsight, that would obviously have been the most efficient, but it needs trust and cooperation between regional Stasi offices to give up control of their records.
But The Stasi did not have foresight, was not efficient, did not have cooperation internally. (If they had all those attributes things may have gone differently. So it’s good they were still incompetent at the end)
The snippets contain sensible data of Stasi victims and often false statements about them. There is a specific law governing the handling of Stasi files, regulating who can get access to what.[1]
Wow, fascinating! I hope I’m not the only OCR-minded soul on here that immediately started thinking of pipelines for this one could assemble from all our new DL-enabled toys… it’s a shame that “some American kid” probably doesn’t meet their contracting requirements.
One thing that kinda baffles is that it sounds like the snippets (I bet that German word is fun) haven’t been scanned(/photographed…?) yet. No matter who they contract with, wouldn’t that necessarily be step #1? Or is the scanning the part that needs contracting, and the “e-Puzzler” name was just a lil melodramatic?
For one thing, I feel like it’ll be wildly helpful that they’re on a variety of papers, not challenging like they imply. Once that and bag index have narrowed down neighbor candidates, you’ve got rules and glyphs to go off of, much less recognizable form layouts. It feels like the Vesuvius scrolls but actually tractable…
16k bags with 3k shreds each is a few million pieces. You could scan 10-20, or more, at a time spreading a them over a glass sheet, putting other on top and making two pictures. Have a few people doing that and you can process a bag in a few hours. 16k bags for a team that can process 10 bags in parallel is like a year of work.
After that I bet the full reconstruction takes like a month tops with AI matching pieces.
That’s an odd price. Not knowing what a “bag” means, I think 13-gallon (“kitchen”) or larger lawn clippings bag is possible.
I wouldn’t for €100 sign up to scan even an office wastebasket (6-8 gallon) bag of torn-up paper. Make it €2000 for that quantity, and we’ll start talking.
I worked in the Pentagon from 1989-1995. I handled “burn bags”, among the other classified stuff I did.
I can’t speak for what their bags are like, but ours were basically large, thick, grocery-size paper bags without handles. Thick enough that they most likely wouldn’t tear, even if they were full and mid-handled somewhat. You rolled down the top and stapled them shut.
But we didn’t shred our documents first. We just put in whole sheets and reams of paper, and we left it up to them to shred first and then burn. But then, that was 1989.
Don't neglect the possibility that there are people in power who don't want these records to come to light. Either because of direct implication of people or their families in the stasi atrocities, "national embarrassment", etc
> Could shredded paper be reconstructed?
> No. A few thousand other bags contained paper that had been shredded into incredibly small pieces in 1990. This material was destroyed back in 1991 after inspection by the Stasi Records Office. Hence, there is no longer any paper shredded by the Stasi at today's archive .
I find the assertion that it was “impossible” to reassemble shredded documents… disappointing - as they seem to be confusing “very difficult with current tools” with “no way of doing it, ever”
I mean, they made sure it was true by burning them all after they decided it could never be done. Phenomenally short-sighted - unless of course there was a reason they wanted those particular documents to never be read.
That's the bags they kept, the sentence is about "properly" shredded bags. Which, again, would be possible in 2011 (DARPA shredder challenge) but it's not that unreasonable that 20 years before that it was considered impossible.
> unless of course there was a reason they wanted those particular documents to never be read.
Why would officers from West Germany not want East German documents to be read? That's nonsense. Then they would have destroyed everything, not just th shreddered documents.
Why then would they destroy them, and make sure it’s impossible to ever know their contents. The cost of storage of sacks of paper can hardly have been the reason.
Because they assumed they were already destroyed beyond any possible means of recovery. Would you store a bag of paper pulp/ash that came out of the NSA's pulper/incinerator?
> Since the project began in 2007, based on contractual obligations, around 6.5 million euros have been paid to Fraunhofer IPK or the Fraunhofer Gesellschaft
> In the fall of 2013, Fraunhofer IPK was able to demonstrate that the software developed by its scientists, the so-called e-Puzzler, works in principle. However, the process proved unsuited for mass production. Overall, the project never achieved the envisaged automated reconstruction rate of at least 80 per cent.
What is up with this cynical comment? They tried to solve the problem and it turned out to be not feasible. Just because they failed doesn't mean trying was bad. I suggest learning about the concept of "expected value".
If you knew anything about the European funding landscape, you'd know that a 6 million Euro project is an enormous amount of money for a project with only one institute involved.
Compare that for instance with the Horizon 2020 project "RePAIR – Reconstructing the Past: AI and Robotics Meet Cultural Heritag", which ran for almost 5 years, had a budget of only 3.5 million Euros and that money was divided among six(!) different partners.
Maybe that gives a little bit of a perspective on "expected value".
Hard to get too worked up about that when the US govt has allocated about 500 billion to Lockheed Martin for a jet fighter. I'd honestly put the jet and the docs at about the same expected value ("what exactly is this doing for our people?").
2020 is a significantly later date, meaning the fast moving field of machine learning was in a much better state by then. Moreover, the project you mentioned might have been far smaller in scope. The scope of the Stasi document reconstruction is anything but small. They talk about "400 to 600 million fragments which represent around 40 to 55 million pages."
> No. A few thousand other bags contained paper that had been shredded into incredibly small pieces in 1990. This material was destroyed back in 1991 after inspection by the Stasi Records Office. Hence, there is no longer any paper shredded by the Stasi at today's archive .
Shredder chads from the 90s can definitely be reconstructed. This is why the NSA has gradually decreased the maximum size permissible for chads. If they'd held on to that material they could be recovered today.
Given how much of a hard time they're having with hand-torn pages, which should be a long solved problem, it's very likely that it would have been impossible in practice, in the government IT framework for a long time...
https://archive.org/details/DocumentsFromTheUSEspionageDen/D...
> Could shredded paper be reconstructed?
Apparently! Vol. 32 of above (per a spot check) has many docs that were shredded. My recommended method is to put papers in a bucket of liquid and then smush it. Good luck reconstructing that.
For fast emergency destruction of documents, the NSA has developed a procedure https://media.defense.gov/2021/Jul/02/2002755769/-1/-1/0/EME... where paper and oxidizer get layered in a burn barrel. "The ratio which was adopted was 65 pounds of paper for 100 pounds of nitrate" gives you an idea how much extra oxygen is required for proper combustion.
In hindsight, that would obviously have been the most efficient, but it needs trust and cooperation between regional Stasi offices to give up control of their records.
But The Stasi did not have foresight, was not efficient, did not have cooperation internally. (If they had all those attributes things may have gone differently. So it’s good they were still incompetent at the end)
[1] Gesetz über die Unterlagen des Staatssicherheitsdienstes der ehemaligen Deutschen Demokratischen Republik (Stasi-Unterlagen-Gesetz - StUG), https://www.gesetze-im-internet.de/stug/BJNR022720991.html (in German)
One thing that kinda baffles is that it sounds like the snippets (I bet that German word is fun) haven’t been scanned(/photographed…?) yet. No matter who they contract with, wouldn’t that necessarily be step #1? Or is the scanning the part that needs contracting, and the “e-Puzzler” name was just a lil melodramatic?
For one thing, I feel like it’ll be wildly helpful that they’re on a variety of papers, not challenging like they imply. Once that and bag index have narrowed down neighbor candidates, you’ve got rules and glyphs to go off of, much less recognizable form layouts. It feels like the Vesuvius scrolls but actually tractable…
After that I bet the full reconstruction takes like a month tops with AI matching pieces.
I wouldn’t for €100 sign up to scan even an office wastebasket (6-8 gallon) bag of torn-up paper. Make it €2000 for that quantity, and we’ll start talking.
I can’t speak for what their bags are like, but ours were basically large, thick, grocery-size paper bags without handles. Thick enough that they most likely wouldn’t tear, even if they were full and mid-handled somewhat. You rolled down the top and stapled them shut.
Like these: https://live.staticflickr.com/4129/4987369106_ab71780b90_z.j...
But we didn’t shred our documents first. We just put in whole sheets and reams of paper, and we left it up to them to shred first and then burn. But then, that was 1989.
Agreed, €100 seems low to process a single bag.
I assume both scanning and reconstruction would be part of the same contract.
I speculate the problem is not technical but financial or political; the ML to do this was up to the task at least ten years ago.
https://cacm.acm.org/news/darpa-shredder-challenge-solved/
I mean, they made sure it was true by burning them all after they decided it could never be done. Phenomenally short-sighted - unless of course there was a reason they wanted those particular documents to never be read.
Why would officers from West Germany not want East German documents to be read? That's nonsense. Then they would have destroyed everything, not just th shreddered documents.
Talk about rearranging deck chairs on the Titanic.
> In the fall of 2013, Fraunhofer IPK was able to demonstrate that the software developed by its scientists, the so-called e-Puzzler, works in principle. However, the process proved unsuited for mass production. Overall, the project never achieved the envisaged automated reconstruction rate of at least 80 per cent.
Well, that was tax-payer's money well spent!
Compare that for instance with the Horizon 2020 project "RePAIR – Reconstructing the Past: AI and Robotics Meet Cultural Heritag", which ran for almost 5 years, had a budget of only 3.5 million Euros and that money was divided among six(!) different partners.
Maybe that gives a little bit of a perspective on "expected value".