Forum adverts like this one are shown to any user who is not logged in. Join us by filling out a tiny 3 field form and you will get your own, free, dakka user account which gives a good range of benefits to you: No adverts like this in the forums anymore. Times and dates in your local timezone. Full tracking of what you have read so you can skip to your first unread post, easily see what has changed since you last logged in, and easily see what is new at a glance. Email notifications for threads you want to watch closely. Being a part of the oldest wargaming community on the net. If you are already a member then feel free to login now.

 Ensis Ferrae wrote:

nfe wrote:

6. Again, what do you mean by pattern match?

My guess here is, you take tablets/artifacts from a given civilization that we have not deciphered their language yet, pattern matching algorithms would be able to much more quickly scan an entire catalogue of artifacts than any individual person is. Finding a pattern may give a clue to syntax or grammar of the given culture which may point researchers to that "aha!" moment where they are then able to decipher it.

That is not how it works I am afraid. Certainly, we can pick out patterns like "this character occurs very often in certain positions, it may represent a vowel". However, the patterns do not tell us anything about the meaning. We still do not know what vowel the character represents. Furthermore, the catalogue of texts from most undeciphered languages is not that big, so a human can easily do the same work at a lower cost.

As I understand it, there is only one artefact with Linear A on it, which has already been intensively studied and not yielded the secrets of the language.

Leaving aside the argument about laser scanning versus photography, what are the most important ancient languages to try and translate with new methods?

I think the Harappan script deserves a lot of study.

 Kilkrazy wrote:
As I understand it, there is only one artefact with Linear A on it, which has already been intensively studied and not yielded the secrets of the language.

Leaving aside the argument about laser scanning versus photography, what are the most important ancient languages to try and translate with new methods?

I think the Harappan script deserves a lot of study.

I am not sure as of how many artefacts with Linear A on them are there, but it is definitely more than 1. As of 2000, there were 1427. However, most of the inscriptions are very short so the total extent of all Linear A text known in 2000 was only 7,362 to 7,396 characters (which would fit on two sheets of paper).

To flip the argument and look at the reverse, suppose an alien archaeology team came to Earth after a major collapse and had to decipher the meaning of the Latin alphabet.

Would it help or hinder the project, that there are millions of pages of Latin script but they are in dozens of different languages?

Let's assume all the pages which have a German-Russian-Japanese translation (the Rosetta Stone of this scenario) have vanished for some reason.

If we expand this idea of digitizing artifacts a bit, and pattern recognition and whatnot a bit, I think it really is something of an idea that has merit, not least of which because of donkey caves like ISIS and others who would intentionally (or not) destroy artifacts, thus losing that knowledge forever.

But, there's also something to be said for the numerous studies that show how your native current language affects how you think. . . By digitizing, researchers who otherwise have no other means/opportunities to study a piece would be able to, and just the difference in thought processes may unlock some new insights.

Further, in the realm of pattern recognition, lets say we apply it to known languages and known bodies of work that may (or may not) show some slight differences in works wherein we may gain further insight into common people's lives in terms of vernacular or slang used vs. governmental "official" language. Obviously with no time machines, we cannot say for certain that slang is being used or vernacular changes mean that a tablet that we think is an order for a shipment of fish from the market, is really a 1 star yelp review (maybe a bit extreme, but I hope you get what I'm trying to say).

 Kilkrazy wrote:
To flip the argument and look at the reverse, suppose an alien archaeology team came to Earth after a major collapse and had to decipher the meaning of the Latin alphabet.

Would it help or hinder the project, that there are millions of pages of Latin script but they are in dozens of different languages?

Let's assume all the pages which have a German-Russian-Japanese translation (the Rosetta Stone of this scenario) have vanished for some reason.

That would indeed be an insanely impossible task because the aliens would, rightfully so, assume that each text was the same language. Though it is unlikely that the bronze age had a similar dilemma since even if two wildly different languages had shared characters for their written language that the written languages would follow wildly different rules for the writing. Especially if its a pictographical writing vs our modern phonetic writing. Pictographs directly represent an idea or object. Phonetic writings represent sounds, and those sounds then represent an idea or object. Its a higher degree of separation for people trying to translate the text.

Well, there are variations of Latin alphabets, for example Norwegian and French contain characters or accents which are not found in each other or in English.

However, Linear A and Linear B contain a lot of similar characters, but while Linear B has been deciphered it did not lead to decipherment of Linear A because the actual language of Linear A is completely different to Linear B.

It's like if you use the Latin alphabet to express Japanese, which is mostly possible (Romaji.) There's no relation of structure and grammar between Japanese and Lithuanian.

That said, most European languages are related to each other and have some similar words and structures, so perhaps it would be possible for our imaginary alien archaeologists to get somewhere.

Indeed. Especially since many words are shared, even if their spelling might vary.

You'd probably be able to translate French, Spanish, and Italian fairly easily since they share words and most rules of grammer. English would be more of a stretch, but doable since there are still shared words. Aliens might conclude that all of the Romance languages are a single language with some regional variation in spelling, and that English was a different language that split off from the Romance language. Not exactly what happened, but close enough.

Of course if they found sound recordings on any functional machines that might muddy the waters.

 Grey Templar wrote:
Indeed. Especially since many words are shared, even if their spelling might vary.

You'd probably be able to translate French, Spanish, and Italian fairly easily since they share words and most rules of grammer. English would be more of a stretch, but doable since there are still shared words. Aliens might conclude that all of the Romance languages are a single language with some regional variation in spelling, and that English was a different language that split off from the Romance language. Not exactly what happened, but close enough.

Of course if they found sound recordings on any functional machines that might muddy the waters.

How would they be able to translate French or Spanish in the first place though?
All they would see is a bunch of incomprehensible characters, and while they might be able to recognise patterns, without a reference point (a translation into a language they do know) they will never be able to figure out the meaning of those patterns. Here, try to tell me what this means without using an intermediary translation (so no Google Translate):

நீ இதை முயற்சி செய்கிறாய் என்று எனக்கு தெரியும்

Or for another challenge, try figure out the meaning of this short phrase. It is in the Latin alphabet, but it is in a made-up language (based on a variety of European languages with unique grammar and syntax) so you won't find a translation even if you secretly try :

Maj jucha tivoj kajara jamitäl.

The first one would be the challenge faced by alien archaeologists and linguists trying to decipher Earth languages. Unknown characters, an unknown language, and they can't look up a translation into a language they do know. The second one is like the challenge faced by people trying to translate Linear A. We know the writing system, just not the language it records. And Linear A is actually more difficult because unlike my fictional language here it does not appear related to any other known language.

It looks a bit like Esperanto.

That is based on having read the Stainless Steel Rat books, in which it's used, and having started to study it about 25 years and got distrracted and learned Japanese instead.

Yup, without context you have nothing to go off of. You'd be reliant on finding an artifact labeled with its name or something to do with it. Like maybe a store catalogue with labeled pictures, thus allowing you to translate some specific key words which you could use to build up some contextual links. Like translating Axe because you found tons of pictures of Axes with that word next to them. You then look for Axe in other samples of text, which maybe leads you to the word Wood or Tree. Which then leads to other words.

If you have writing and nothing else to put it in context you are basically screwed.

 Iron_Captain wrote:




Or for another challenge, try figure out the meaning of this short phrase. It is in the Latin alphabet, but it is in a made-up language (based on a variety of European languages with unique grammar and syntax) so you won't find a translation even if you secretly try :

Maj jucha tivoj kajara jamitäl.

That's easy:

Maj jucha tivoj kajara jamitäl. translates to:

"Iron Captain was here"

nfe wrote:

I'm not worried about jobs. I'm not a philologist and they'd all keep their jobs to interpret and write commentaries anyway. If I thought it was going to help get more done I'd be all for it. I don't think laser scanning artefacts gains us anything. At least not at the present cost and time involved.

1. I can do this with RTI and photogrammetry.
2. See 1.
3. See 1.
4. RTI and photogrammetry isn't 2d.
5. Not without vast, vast amounts of work.
6. Again, what do you mean by pattern match?
7. See 1.

nfe wrote:

1. I can do this with RTI and photogrammetry.

No you cant, not to the same level of precision. Laser scanners make a 3d image down to a minute level. You can only do analysis from the photograph, with photogrammetry with multiple photographs you can have a rough indication but that is all.
You can also work a lot faster from a digital 3d sculpt, zoom freely rotate etc. Photogrammetery allows very limited image rotation and positioning. Ironically you need more specialist equipment for photogrammetery than for accessing 3d images, for which you only need a powerful enough graphics card.
By the way you said photographs initially, which even more rigid.

nfe wrote:

2. See 1.
3. See 1.

Compounded error

nfe wrote:

4. RTI and photogrammetry isn't 2d.

It is, more or less, you can extrapolate 3d from it to a limited extent, however a laser scanned image from multiple directions recreated into a digisculpt is true 3d.
Photogrammetry is broadly 3d in the same way (though somewhat better) as 3d movies with special goggles are 3d.

Photogrammetery is useful, its a technology born of aerial reconnaissance, and fiurst used widely with good results from the first world war. Normally its quite hands on, and you need access to precise photographs and a bioptics. Which is a mess compared to a 3d sculpt.

nfe wrote:

5. Not without vast, vast amounts of work.

Nope. Feed it into a codebreaker. Let a supercomputer do the pattern matching. As its a 3d digital image to start with and not some backward hands-on photogrammetry its works as fast as you can load data.
You will need to outsource this, but if the project can buy the hardware various security agencies can provide the software, and cleared ex-employees handle the data processing. Archeologists handle the tablets and provide the feed of 3d laser scan data. Unlikely they will be given access to the codebreaker itself, for obvious reasons, but the filtered output file is all yours.

nfe wrote:

6. Again, what do you mean by pattern match?

The way most codebreaking computers work, look for patterns in the data, literally. In this case look for patterns of combinations of symbols. It helps if you have an idea where a tablet comes from. A a digital process human operators can add guesses. So for a pattern in a code you might be looking for the letter E. With tablets you might see if you get a match if you assume a certain set of symbols in a batch of tax records means 'grain' if this doesnt make sense you an try against with 'cows'. etc etc. It works by assigning values to pattern matches until groups of values make sense, which the codebreaker does for you.

nfe wrote:
7. See 1.

Interesting how both of us are looking at using proliferated military technology to remote analyse clay tablets. Yet from opposite ends of the twentieth century.

It never ceases to amaze me how often people on the internet turn out to be experts in absolutely any subject field which they care to turn their hand to. Truly we live in an age of wonders.

Orlanth so your whole idea hinges on super codebreaking computers used at government/military level. So that's the first massive barrier against getting any kind of access to that kind of sensitive software - not to mention even if you can get access the costs would be big. Archeaology is not swimming with money - it doesn't generate huge profits from huge investments. It's money invested into knowledge and relies heavily on donations, government legislation, institutional budgets and such.


Also you are seriously overlooking the fact that codebreakers are working within known languages; dead languages are, well, dead. If there's no common known base (like latin) then the code breaker has nothing or very little to work with. Not to mention the fact that language is not just a straight case of translating one word into another word.

Often many languages have words, concepts and meanings that are totally missing from other languages and it takes time to work out the subtle meanings (esp as those are often not detailed in written records).
Just look at how Google Translate mangles current languages when it translates. Yes it can sort of get the gist of things sometimes, but it really mangles things most of the time and fails on more complex or subtle use of language.

You seem to want to argue about this. I'm not sure why. I'm all for laser scanning everything. Why not? The more forms of record the better. However, it is not currently viable, nor is it the most efficient approach.

 Orlanth wrote:

nfe wrote:

1. I can do this with RTI and photogrammetry.

No you cant, not to the same level of precision. Laser scanners make a 3d image down to a minute level. You can only do analysis from the photograph, with photogrammetry with multiple photographs you can have a rough indication but that is all.
You can also work a lot faster from a digital 3d sculpt, zoom freely rotate etc. Photogrammetery allows very limited image rotation and positioning. Ironically you need more specialist equipment for photogrammetery than for accessing 3d images, for which you only need a powerful enough graphics card.
By the way you said photographs initially, which even more rigid.

No, not with the same level of precision, but with ample precision for the task. Especially with RTI. This is what we are actively doing with tablets now. It's what our philologists want us to do if they even want anything more than simple photographs. The latter is plenty in most circumstances. I also have complete rotation with both.

nfe wrote:

4. RTI and photogrammetry isn't 2d.

It is, more or less, you can extrapolate 3d from it to a limited extent, however a laser scanned image from multiple directions recreated into a digisculpt is true 3d.
Photogrammetry is broadly 3d in the same way (though somewhat better) as 3d movies with special goggles are 3d.

Again, no, it is not as accurate as laser scanning. The images created from both RTI and photogrammetry are 3D, however, and RTI is becoming the preferred approach even by people who can afford to laser scan (there are a couple PhDs being written currently, supervised by leaders in the fields both from the archaeological and imaging sides, by people working on Pictish stones specifically dealing with RTI as an ideal (at current levels of technology) approach. I can put you onto them. We share an office.

Photogrammetery is useful, its a technology born of aerial reconnaissance, and fiurst used widely with good results from the first world war. Normally its quite hands on, and you need access to precise photographs and a bioptics

You need a lot of mediocre photos and a computer running Agisoft photoscan. You drag and drop the images into photoscan, run a batch process to align, build a dense cloud, build a mesh, and texture it, then let it do its thing. That's it. I do actually do this for major archaeological projects.

nfe wrote:

5. Not without vast, vast amounts of work.

Nope. Feed it into a codebreaker. Let a supercomputer do the pattern matching. As its a 3d digital image to start with and not some backward hands-on photogrammetry its works as fast as you can load data.

That's not the work I mean. I mean the philological work to provide the basis for everything else.

You will need to outsource this, but if the project can buy the hardware various security agencies can provide the software, and cleared ex-employees handle the data processing. Archeologists handle the tablets and provide the feed of 3d laser scan data. Unlikely they will be given access to the codebreaker itself, for obvious reasons, but the filtered output file is all yours.

No one has the money for this. I think you night be overestimating the budgets projects run on. One of the ones I'm on is one of the largest (might actually be THE largest) project in the Near East and we run on c.35k a year. That's the project, not anyone's individual salary. Another I work on is funded by an Ivy League university, a Russell Group university, and has a serious grant from the UK government and runs on a fraction of that.

nfe wrote:

6. Again, what do you mean by pattern match?

The way most codebreaking computers work, look for patterns in the data, literally. In this case look for patterns of combinations of symbols. It helps if you have an idea where a tablet comes from. A a digital process human operators can add guesses. So for a pattern in a code you might be looking for the letter E. With tablets you might see if you get a match if you assume a certain set of symbols in a batch of tax records means 'grain' if this doesnt make sense you an try against with 'cows'. etc etc. It works by assigning values to pattern matches until groups of values make sense, which the codebreaker does for you

So what humans have already done. This could be useful if we unearth a vast corpus of an entirely new language (which would be a big enough deal tht you might actually manage to get a grant to scan it all!), but little use in usual circumstances when texts are found in very small numbers over decades. With the nature of excavation in modernity, no one is about to find another Kültepe with several thousand tablets in a season.

Again, as I've said, I'm all for recording everything in as many ways as possible. I'm one of very few surveyors left who still make plans with handheld and drone photogrammetry, total stations and CAD, and handdrawings. My problem is with the practicality, economic viability, and relative efficiency of a technology.

So, I read 1177, if I wanted to learn more about the Late Bronze Age as a layman, where should I go next?

nfe wrote:

You will need to outsource this, but if the project can buy the hardware various security agencies can provide the software, and cleared ex-employees handle the data processing. Archeologists handle the tablets and provide the feed of 3d laser scan data. Unlikely they will be given access to the codebreaker itself, for obvious reasons, but the filtered output file is all yours.

No one has the money for this. I think you night be overestimating the budgets projects run on. One of the ones I'm on is one of the largest (might actually be THE largest) project in the Near East and we run on c.35k a year. That's the project, not anyone's individual salary. Another I work on is funded by an Ivy League university, a Russell Group university, and has a serious grant from the UK government and runs on a fraction of that.

Oh wow. 35k? A year? Compared to the projects they let me go to you guys are as rich as Kroesus...
The last project I went to we did not even have enough money for an excavator and we had to do all the trench digging by hand. Through clay.
And all we got for our effort was two measly pot shards (it is at moments like this that I start to doubt my career choice ). They weren't even nice pot shards.


Also, Orlanth, what would you want with a codebreaker? Either a text is already known and the only thing left is for linguists to argue over the interpretation and precise translation, or a language is untranslatable because there is no point of reference that a human or computer could use to start deciphering it. There is very little code to break either way.

Easy E wrote:So, I read 1177, if I wanted to learn more about the Late Bronze Age as a layman, where should I go next?

Any particular region or the general Mediterranean? Cyrian Broodbank's Making of the Middle Sea is a great one for the latter. Not exclusively LBA but does a great overview pf the entire international age. There's also a Met Museum book you can download free from their website called Beyond Babylon. It's actually an exhibition book but it has lots of decent short articles.

Iron_Captain wrote:
Oh wow. 35k? A year? Compared to the projects they let me go to you guys are as rich as Kroesus...
The last project I went to we did not even have enough money for an excavator and we had to do all the trench digging by hand. Through clay.
And all we got for our effort was two measly pot shards (it is at moments like this that I start to doubt my career choice ). They weren't even nice pot shards.

We're the traditional site of David and Goliath's battle and mentioned in Sennacherib's invasion of the Shephela so well placed for donations. We don't hire workers anyway.

Who cares about 'nice' sherds? Read them for chronology then throw them away. I'm not really a ceramics guy

 Ketara wrote:
It never ceases to amaze me how often people on the internet turn out to be experts in absolutely any subject field which they care to turn their hand to. Truly we live in an age of wonders.

Just doing what I actually do.

Thanks for the veiled patronising remarks.
If you cant argue with it, denigrate it.

 Overread wrote:
Orlanth so your whole idea hinges on super codebreaking computers used at government/military level. So that's the first massive barrier against getting any kind of access to that kind of sensitive software - not to mention even if you can get access the costs would be big. Archeaology is not swimming with money - it doesn't generate huge profits from huge investments. It's money invested into knowledge and relies heavily on donations, government legislation, institutional budgets and such.

You would be surprised but this sort of thing makes security agencies look good.
It would not be the first time government agencies have done odd work for charity/scientific concerns.
And this is isnt even going that far. What you need but cant have direct access to is the codebreaker software.

Building the supercomputer isn't actually as expensive as it sounds, you can make a decent data cruncher with a rack of high end graphics cards, Titan X or better. Cost is about £25k rather than the millions it used to be.

As for the expertise, yes it can be garnered, the archeological project would have to be government backed, and the same government can clear the staffing.

 Overread wrote:

Also you are seriously overlooking the fact that codebreakers are working within known languages; dead languages are, well, dead. If there's no common known base (like latin) then the code breaker has nothing or very little to work with. Not to mention the fact that language is not just a straight case of translating one word into another word.

Not overlooking that at all, codebreakers have to work with raw pattern as you aren't looking to translate into neat sentences all the time. Also there is an advantage, the ancient Hittites aren't trying to hide things, people who double encrypt are.
Also codebreaking is made easier with a larger dataset. The more info in the dataset the easier it is to find a pattern. This is why codes are changed often even if double encrypted.

Point is there are professional agencies who work with decoding encrypted data. Current encryption technologies are so advanced now, even in the public domain that the process of decrypting is near impossible, yet cyrptoanalysis departments are a big factor in current intelligence.
You and I don't know exactly what government agencies have available, but if its set up to handled PGP or equivalent, its will find clay tablets a doddle. Hittites didn't have double encryption, in fact it isn't encrypted at all, so the pattern analysis is more straightforward.

There is a staff turnover in places like Langley and GCHQ, and not many places to go onto with the skills people accumulate there. Yes I do think it would be a reasonable suggestion to approach the UK or US government with a plan such as this.

 Overread wrote:

Just look at how Google Translate mangles current languages when it translates. Yes it can sort of get the gist of things sometimes, but it really mangles things most of the time and fails on more complex or subtle use of language.

Google translate is actually a good example, yes it 'mangles' things, but a half mangled message is still a half clear message, a human operator can see that as progress. It isn't a case of input data press button, you will need to analyse the data too.

nfe wrote:
You seem to want to argue about this. I'm not sure why. I'm all for laser scanning everything. Why not? The more forms of record the better. However, it is not currently viable, nor is it the most efficient approach..

Why do you say that. How do you store your photographs. Filing cabinet, even if you make an image database it will be less data efficient than a 3d digital image, you need binocular photphraphy to make 3d images yes?

nfe wrote:

No, not with the same level of precision, but with ample precision for the task. Especially with RTI. This is what we are actively doing with tablets now. It's what our philologists want us to do if they even want anything more than simple photographs. The latter is plenty in most circumstances. I also have complete rotation with both.

But its hands on work at human speed.

nfe wrote:

Again, no, it is not as accurate as laser scanning. The images created from both RTI and photogrammetry are 3D, however, and RTI is becoming the preferred approach even by people who can afford to laser scan (there are a couple PhDs being written currently, supervised by leaders in the fields both from the archaeological and imaging sides, by people working on Pictish stones specifically dealing with RTI as an ideal (at current levels of technology) approach. I can put you onto them. We share an office.

I would love to meet these people.
But again from your work its still mostly analogue. The main advantage of fully digitising the images to high quality (laser) is to allow a computer to directly analyse them. Now computer can analyse photographs, but its far less fluid, you get background contamination, lighting etc.

nfe wrote:

You need a lot of mediocre photos and a computer running Agisoft photoscan. You drag and drop the images into photoscan, run a batch process to align, build a dense cloud, build a mesh, and texture it, then let it do its thing. That's it. I do actually do this for major archaeological projects.

The database of compiled images, how easy is it to cross reference. Can that be automated with your available technology?

Just because it what you actually do on your admitted shoestring doesn't mean its not crude, you take your lot of medicore images and you en up with a 'google map' effect, its not actual 3d, you can rotate from certain points of view, a 3d sculpt has true rotation, and with images you need to pick out what you can pick out. If you miss a symbol amongst cracks a computer wont necessarily do the same if given an accurate enough dataset.

Nope. Feed it into a codebreaker. Let a supercomputer do the pattern matching. As its a 3d digital image to start with and not some backward hands-on photogrammetry its works as fast as you can load data.

That's not the work I mean. I mean the philological work to provide the basis for everything else.

Actually the computer will help there also. You categorise the launguages based on two factors, where the artifact was found and what was written on it. The data on the tablet can be categorised by the computer depending on what patterns it detects, location data is on the file.

No one has the money for this. I think you night be overestimating the budgets projects run on. One of the ones I'm on is one of the largest (might actually be THE largest) project in the Near East and we run on c.35k a year. That's the project, not anyone's individual salary. Another I work on is funded by an Ivy League university, a Russell Group university, and has a serious grant from the UK government and runs on a fraction of that.

That's the way the minds are channelled. The ideology of tweedy nerds in a back office or a camper van with a shoe string. It need not be that way.
Have you even thought about approaching a codebreaking organisation for help, does it even occur to you. It would be a blessing in PR for somewhere like GCHQ to provide expertise to make a breakthrough on ancient languages, more than worth their money.

nfe wrote:

So what humans have already done. This could be useful if we unearth a vast corpus of an entirely new language (which would be a big enough deal tht you might actually manage to get a grant to scan it all!), but little use in usual circumstances when texts are found in very small numbers over decades. With the nature of excavation in modernity, no one is about to find another Kültepe with several thousand tablets in a season.

At a tiny fraction of the speed and efficiency. Also humans cant cross reference 10000+ clay tablets AT THE SAME TIME, looking for pattern matches.
no wonder you dont make much headway.
As for finds, from what I heard the majority of Hittite era tablets haven't even been catalogued yet.

nfe wrote:

Again, as I've said, I'm all for recording everything in as many ways as possible. I'm one of very few surveyors left who still make plans with handheld and drone photogrammetry, total stations and CAD, and handdrawings. My problem is with the practicality, economic viability, and relative efficiency of a technology.

You think, but do you think outside the box?

Getting funding could be as easy as convincing government that unlocking history of the Bronze Age collapse might give useful datasets about societal collapse itself and how it works. There is the possibility we are due one. Plus the ancillary benefits of asking for help with a problem that becomes a feather in the cap if solved. 'Langley/GCHQ decrypted an ancient language causing a leap forward in understanding' helps offset 'Langley/GCHQ is reading peoples emails'.

Let me give you an example. There are forestry projects on West Falkland set up as a model as to how to revitalise a 'wasteland'. How do you get funding to plant trees. Either you think in the box get a tiny pittance of a grant and slow on painfully slowly. Or you don't, you ask the army because they have soldiers on station that can do that for you in a tiny faction of their ample spare time, and seed can be brought in through normal shipment on your miniscule budget, or you can have it send though with some fresh strawberries and a bottle of brandy on the next Hercules to arrive at Mt Pleasant. What is a theoretical ecological project becomes realised by outside the box thinking and approaching a completely unrelated government department.

You have nothing to lose by trying and in case the closed minds prevail, archeology has already in the past had assistance from military codebreakers. You would not be the first one to ask.

 Orlanth wrote:


Google translate is actually a good example, yes it 'mangles' things, but a half mangled message is still a half clear message, a human operator can see that as progress. It isn't a case of input data press button, you will need to analyse the data too.

Well yes, except that Google translate is mangling a result from two known languages with huge bodies of data from both languages as well as bodies of data of translation from both languages. With a dead language you've got a pittance of information in comparison, plus no actual translation structure. You don't even know fully how the language is built up at the ground level to know how they are forming thoughts, objects and actions into symbols.

So if Google is a good example then the computer tech has a long long way to go before it can be ready to collate and event attempt to translate vast bodies. It can certainly speed things up, certain areas can be sped up, but it would seem that the use of a human operator is still critical to the actual translation at this stage.


Also a lot of research groups and such do think outside the box, but sometimes the box itself can have restrictions on what they can and cannot do. Plus there's a big difference between convincing the army to plant trees on their day off through to gaining access to military grade computing. Not saying its not possible, its just not an easy approach considering that there's already a system in place that does work.

 Iron_Captain wrote:



Also, Orlanth, what would you want with a codebreaker? Either a text is already known and the only thing left is for linguists to argue over the interpretation and precise translation, or a language is untranslatable because there is no point of reference that a human or computer could use to start deciphering it. There is very little code to break either way.

Fair question.

People think codebreakers work exclusively with text. They do not. Its just the modal function. Codes can be symbols, and clay tablets are covered in symbols.
Most codebreaking (or at least most of what we know of codebreaking that isn't heavily classified) involved pattern recognition. there is a lot more to that, but because clay tablets are not computer encrypted, or in fact encrypted at all, we only need to deal with the more elementary elements of codebreaking (and thus within my amateur set of knowledge on the subject).

What you need to break an elementary pattern code is a good guess. So for example if the clay tablets are likely tax records you will want to highlight symbols that refer to possible numbers, and then tally accompanying symbols to see if any could mean various products. There was no coinage then but taxes might have been paid in grain or cows or something else. By having the dataset of many thousand digitised clay tablets to hand in the database a decent computer could categorise all occurances of any particular symbol, where they appear on which tablet and what symbols they appear next to. Rare symbols can be isolated from common ones. If there is a symbol that appears exactly once on every tablet of a sequence of tables, its likely some sort of header or title, or the name of the recipient or scribing office.

The volume of data helps here, the more tablets there are digitised the more the patterns will come through. one clay tablet will tell you zip, but 10000 or more could tell you quite a bit. You then need to make intelligent guesses as to meaning, see above, but hold them lightly. Thankfully with a computer you can hold onto all your guesses in a separate accompanying file. If I guess right a decent cryptoanalysis software package now does some of this for you, but that is a guess on my part. In any event you can database the assumptions of likely symbol meaning based on the study of a large volume of tablets, and cross reference the assumptions to look for pattern matches on a higher level, pattern matches of assumed meaning. If you have what you think are numbers interspaced by what you think are nouns you likely have an inventory of some description.

These are languages, so there will be patterns, eve something as elementary as determining whether you are looking at a tablet with a inventory, tax or otherwise, or a poem from the arrangement of the yet untranslated symbols is useful.

The bad news is that to make a breakthrough by pattern analysis you have to make several breakthroughs at once, because you cant necessarily determine any specific symbol. But you can make intelligent guesses due to patterning, The computer holds those for you and does the second level of pattern matching to look for groups of symbols together than now make sense. As soon as you get even one symbol guessed with a loose degree of accuracy (it can still be wrong - in which case you start again and notify the assumption as likely incorrect) you can look for a larger pattern.
Numbers/numerical data are likely the first things to be looked for, and should be very common in some types of document, and largely absent from others.

Remember to have this study to be of any use at all you will need a large data seed, as many clay tablets as you can get your hands on from the same period and culture, all digitised by the same methodology so the computer can assess catalogue and cross reference them automatically.


Automatically Appended Next Post:

 Overread wrote:

 Orlanth wrote:


Google translate is actually a good example, yes it 'mangles' things, but a half mangled message is still a half clear message, a human operator can see that as progress. It isn't a case of input data press button, you will need to analyse the data too.

Well yes, except that Google translate is mangling a result from two known languages with huge bodies of data from both languages as well as bodies of data of translation from both languages. With a dead language you've got a pittance of information in comparison, plus no actual translation structure. You don't even know fully how the language is built up at the ground level to know how they are forming thoughts, objects and actions into symbols.

Now you are dealing with a non alphabetical code, because that is what you re describing. To a codebreaker thats the work of any day ending in 'y'.
This is why I suggest looking at codebreaking, not translation.

 Overread wrote:

So if Google is a good example then the computer tech has a long long way to go before it can be ready to collate and event attempt to translate vast bodies. It can certainly speed things up, certain areas can be sped up, but it would seem that the use of a human operator is still critical to the actual translation at this stage.

Google translate is a good laymans example of what a computer is trying to do. one that readers can experiment with themselves.
Dont assume that the complexity levels or are comparable.

 Overread wrote:

Also a lot of research groups and such do think outside the box, but sometimes the box itself can have restrictions on what they can and cannot do. Plus there's a big difference between convincing the army to plant trees on their day off through to gaining access to military grade computing. Not saying its not possible, its just not an easy approach considering that there's already a system in place that does work.

Good then you will have no problems with the idea that there is no harm in asking. Places like GCHQ are very busy, and wont want to help directly, but they might put you in contact with ex staff through the proper intermediary. Its not the sort of thing you ask at the front desk, unless you want a boatload of CSR denyspeak. I think the best way forward would be to find an amateur archeology buff who is serving in the UK or US armed forces. Such a person could speak to someone from military intelligence by open contact without controversy. Work a way in from there.
Point in our favour is that there is not a lot of work someone who is ex-Langley or GCHQ codebreaker can do, its a very esoteric skill. Running a clay tablet codebreaker computer is work for some guy who needs it. Yes I can see agency staff being very positive to projects like this.

 Orlanth wrote:

 Ketara wrote:
It never ceases to amaze me how often people on the internet turn out to be experts in absolutely any subject field which they care to turn their hand to. Truly we live in an age of wonders.

Just doing what I actually do.

Thanks for the veiled patronising remarks.
If you cant argue with it, denigrate it.

I've seen you argue as if you were an expert on everything from historical military affairs, to obscure religions, to contemporary politics, to the cultural makeup of ethnic groups, to convoluted international diplomatic problems, to aquatic biology, to legal minutae.

And now it would seem, we can add the finer points of decrypting archaeological remains to extract lost languages to your skillset. Hell, you're so very confident about your knowledge on this one in fact, that you seem to be arguing with someone who literally makes a living off it. And now you've started telling us how easy it is to build supercomputers as well in your most recent posts!

Truly, there would seem to be no field of which you are not a master.

I'll respnd fully later, but in the interest of polite conversation, it might be worth thinking about whether you think heads of departments and directors of institutes at world-leading universities with decades of successful funding applications behind them in fields that are built almost entirely on thinking outside the box to get fieldwork done are really people you want to blithley and patronisingly call closed minded when it comes to finding ways of carrying out research.

 Orlanth wrote:

 Iron_Captain wrote:



Also, Orlanth, what would you want with a codebreaker? Either a text is already known and the only thing left is for linguists to argue over the interpretation and precise translation, or a language is untranslatable because there is no point of reference that a human or computer could use to start deciphering it. There is very little code to break either way.

Fair question.

People think codebreakers work exclusively with text. They do not. Its just the modal function. Codes can be symbols, and clay tablets are covered in symbols.
Most codebreaking (or at least most of what we know of codebreaking that isn't heavily classified) involved pattern recognition. there is a lot more to that, but because clay tablets are not computer encrypted, or in fact encrypted at all, we only need to deal with the more elementary elements of codebreaking (and thus within my amateur set of knowledge on the subject).

What you need to break an elementary pattern code is a good guess. So for example if the clay tablets are likely tax records you will want to highlight symbols that refer to possible numbers, and then tally accompanying symbols to see if any could mean various products. There was no coinage then but taxes might have been paid in grain or cows or something else. By having the dataset of many thousand digitised clay tablets to hand in the database a decent computer could categorise all occurances of any particular symbol, where they appear on which tablet and what symbols they appear next to. Rare symbols can be isolated from common ones. If there is a symbol that appears exactly once on every tablet of a sequence of tables, its likely some sort of header or title, or the name of the recipient or scribing office.

The volume of data helps here, the more tablets there are digitised the more the patterns will come through. one clay tablet will tell you zip, but 10000 or more could tell you quite a bit. You then need to make intelligent guesses as to meaning, see above, but hold them lightly. Thankfully with a computer you can hold onto all your guesses in a separate accompanying file. If I guess right a decent cryptoanalysis software package now does some of this for you, but that is a guess on my part. In any event you can database the assumptions of likely symbol meaning based on the study of a large volume of tablets, and cross reference the assumptions to look for pattern matches on a higher level, pattern matches of assumed meaning. If you have what you think are numbers interspaced by what you think are nouns you likely have an inventory of some description.

These are languages, so there will be patterns, eve something as elementary as determining whether you are looking at a tablet with a inventory, tax or otherwise, or a poem from the arrangement of the yet untranslated symbols is useful.

The bad news is that to make a breakthrough by pattern analysis you have to make several breakthroughs at once, because you cant necessarily determine any specific symbol. But you can make intelligent guesses due to patterning, The computer holds those for you and does the second level of pattern matching to look for groups of symbols together than now make sense. As soon as you get even one symbol guessed with a loose degree of accuracy (it can still be wrong - in which case you start again and notify the assumption as likely incorrect) you can look for a larger pattern.
Numbers/numerical data are likely the first things to be looked for, and should be very common in some types of document, and largely absent from others.

Remember to have this study to be of any use at all you will need a large data seed, as many clay tablets as you can get your hands on from the same period and culture, all digitised by the same methodology so the computer can assess catalogue and cross reference them automatically.

I see. But the thing is, all of this has already been done. By hand. Because we do not have a ten thousand clay tablets. The entire corpus of known Linear A texts for example fits on a few sheets of paper. And for many other undeciphered languages we have even less to go on.
We can tell whether some character is a number or a vowel (especially with Linear A, since Linear B which uses the same characters has been deciphered) with a fair degree of accuracy. For Linear A we can even spell out entire words and texts. Problem is those words are unintelligible because they are in an unknown language not closely related to any known language (Minoan is basically the Basque or the Hungarian of the Bronze Age I guess). The only thing we can somewhat guess at is place names (which are presumably derived from the Minoan language and therefore related). For example it may be that the Linear A word "Keniso" is the Minoan name for Knossos. But that is as far as we have gotten in the past 50 years, and as far as we are going to get until we find some bilingual text that could give us a starting point at finding out the meaning behind Linear A words. We can also guess which words are meant to be numbers and which are meant to be commodities, but the precise meaning still eludes us.

On a sidenote, I often do exercises like this where you get a text in an unknown language and then have to answer questions about it based on patterns (I even won a prize once in a competition). Sometimes you can even use patterns to translate an entire text even though it is an incredibly obscure language spoken only by some tribe deep in the Amazon forests. That is pretty cool. Thing is, you always need a starting point. To solve the equation you always need at least one known variable. With dead languages like Linear A, there is none. All variables are unknown which makes them completely impossible to solve or 'break'.

Let's take my phrase again for example:

Maj jucha tivoj kajara jamäl.

And add a few more simple ones:

Maj jucha tivoj kajara obitsjimäl.
Jucha tivoj kajara jamita.
Maj jucha ja isakamäl jamidj.
Jucha ja obitsjimu.
Jucha tivoj kajara alisö obitsjimita.

Just from this small sample, you should be able to figure out which words are verbs, nouns and pronouns, and even what the word order of the language is (whether it is subject-object-verb or subject-verb-object etc.). However, what you will not be able to figure out is what the sentences actually mean. Pattern recognition can only bring you understanding of the structure of a language. To understand what it really means, you need to have at least some translations.

nfe wrote:

Iron_Captain wrote:
Oh wow. 35k? A year? Compared to the projects they let me go to you guys are as rich as Kroesus...
The last project I went to we did not even have enough money for an excavator and we had to do all the trench digging by hand. Through clay.
And all we got for our effort was two measly pot shards (it is at moments like this that I start to doubt my career choice ). They weren't even nice pot shards.

We're the traditional site of David and Goliath's battle and mentioned in Sennacherib's invasion of the Shephela so well placed for donations. We don't hire workers anyway.

I read about that somewhere! so cool.

nfe wrote:
Who cares about 'nice' sherds? Read them for chronology then throw them away. I'm not really a ceramics guy

Ceramics here in Northern Europe are rare enough that they generally get preserved (as long as they are still determinable), although they normally end up in some storage collecting dust since once you have determined them and plotted their location on the map there isn't much you can do with them anymore. It is not like a museum is interested in some small ugly brown-greyish shards. They probably get thrown out at some point, I hope.

 Ketara wrote:

 Orlanth wrote:

 Ketara wrote:
It never ceases to amaze me how often people on the internet turn out to be experts in absolutely any subject field which they care to turn their hand to. Truly we live in an age of wonders.

Just doing what I actually do.

Thanks for the veiled patronising remarks.
If you cant argue with it, denigrate it.

I've seen you argue as if you were an expert on everything from historical military affairs, to obscure religions, to contemporary politics, to the cultural makeup of ethnic groups, to convoluted international diplomatic problems, to aquatic biology, to legal minutae.

I have commented on most of those things, though I don't recall talking about aquatic biology. But then so do you, so do others on Dakka. You dont lay this at them. I have had conversations with the same people on many of the above subjects. Iron Captain included.

Interesting that Iron Captain appears to have some sort of experience on the field of ancient history. I don't doubt him if he claims he has. Yet I also talk with him on subjects as far afield as WW2 history, the politics of the Crimea and other things. I don't see you biting him on the arse for being knowledgeable in multiple fields. Nor should you for the record.

 Ketara wrote:

And now it would seem, we can add the finer points of decrypting archaeological remains to extract lost languages to your skillset. Hell, you're so very confident about your knowledge on this one in fact, that you seem to be arguing with someone who literally makes a living off it. And now you've started telling us how easy it is to build supercomputers as well in your most recent posts!

Yet despite this there are many many topic on Dakka to which I do not respond at all. Fancy that. I thought you assumed I claim to be an expert in everything.

As for arguing with others on a subject. Again this is Dakka.

For the record I have made headway into historical research, with particular note towards medieval weaponry. How did I do this. By being the grand master of medieval knowledge, no, by seeing things others missed. Such as combining facts in a new way. For example recognising that the massed graves at Towton contained longbowmen and men at arms who had evidence of longbow wounds on them that had healed. Wheras historical accounts of the French indicated that a longbow wound was a death sentence. I draw some conclusions from that and hypothesised a theory from those two pieces of data. The theory was shown to have merit for further study by experts in the field.

On an unrelated field I hypothesises how to limit accumulative radiation damage on inhabited space modules. I did this by theorising that is a capillary system of liquid metal was pumped around the outer layers of a space habitat it would absorb radiation before it became embedded deeper in the habitat hull. Accumulative radiation was one of the unfixed problems of long term space habitation. I came up with that not as a noted astrophysicist, I am not, but simply because I think outside the box. That being said I didn't have the skills in chemistry to determine which liquid metal to use. My initial hypothesis involved mercury, but mercury was a poor choice, for purposes of intercepting radiation and cost. NaK on the other hand was ideal and was a cheap material.
Now this has not to my knowledge been implemented anywhere yet, but its one of the problems for long term space habitation that now has a plausible solution. I did that, and I did that as an amateur free thinker.

You will be surprised how much is achieved by outside the box thinking by outside elements. And those who can do this often end up with minor contributions in many fields. It is not particularly rare either, most inventors end up making inventions in several different fields, they just think outside the box in a practical way.

You see I don't need to be an expert in any subject to have insight. But I do have a speciality, what I do is troubleshoot problems with indirect solutions, most notably as a political theorist, but the skillset translates into other fields. Yes I do know a little bit about cryptography, yes I do know a little bit about ancient middle eastern history. Maybe less than others in either field. However I put the two together to hypothesise solutions.

I did this with sufficient reputation for quality that when transcripts of my hypotheses on political matters were presented to Civil Service chiefs they asked for it to be unedited and unsummarised so that the recipients had access to the full nuanced data. From what I hear that is not usual.

nfe wrote:
I'll respnd fully later, but in the interest of polite conversation, it might be worth thinking about whether you think heads of departments and directors of institutes at world-leading universities with decades of successful funding applications behind them in fields that are built almost entirely on thinking outside the box to get fieldwork done are really people you want to blithley and patronisingly call closed minded when it comes to finding ways of carrying out research.

I am not being blithe or patronising. Perhaps you were however.

I respect that you work in the field, I even commented as such. But you on the other hand have come up with the handwave of 'I am the expert, you are not, so off with you'. Blinkers don't suit you
I don't claim to know everything, but when a new idea comes along have you really got the omniscient certainty to write it off. To be honest we see this in academia all the time, its a clash of egos. The 'how dare an outsider think they come up with an valid idea' mindset which causes so many new methods or theories to fail to make traction.

No I am not an expert in your field, but then I don't need to be. I wasn't an expert in medieval weaponry or space structures either, or in the other roles I have contributed to outside of political science. Technically I am not an expert on political science either, I worked some theories which were proven right and went on from there.

Yes directors of institutes at world leading universities tend to know what they are talking about, but the best of them will listen to ideas from the outside. Also sometimes their very expertise may work against them. Acedemicians normally work on direct in field solutions, not indirect out of field solutions, and are often blind to them. Ironically mental structure in formal people groups is one of my actual fields of study, being heavily linked to political theory. Understanding why formal academics and amateur contributors often have completely different ideas for the same field is very similar to why left wing thinkers and right wing thinkers will not draw the same conclusions from the same political data.

Anyway back to the main point. Am I sure that codebreaking will produce results? No I am not. Am I sure that one could approach a codebreaking organisation? That I am more sure of. Could codebreaking produce results? Very possibly, I cannot say for certain because the topic gets classified in a hurry, but codebreaking what-nots can achieve some incredibly impressive results. Doing stuff that is not commonly thought possible. There is nothing I have heard from this thread to guarantee than ancient languages are definitively beyond them.

 Iron_Captain wrote:

I see. But the thing is, all of this has already been done. By hand. Because we do not have a ten thousand clay tablets. The entire corpus of known Linear A texts for example fits on a few sheets of paper. And for many other undeciphered languages we have even less to go on..

If its done by hand it isnt done. People cannot cross reference every symbol and categorise it in relation to other surrounding symbols, it needs a computer.

As for the volume of data:

http://www.britishmuseum.org/about_us/departments/middle_east/facilities_and_services/study_room/studying_cuneiform_tablets.aspx

The British Museum claims to have 130,000 texts. So I don't know where you get the idea of a few sheets of paper.
Looking at the breakdown there are tens of thousands of fragments from some cultures. These are perhaps large enough sample groups for a volume pattern test. Though you will need to know some seed information about where the tablets were found so that you can begin to guess as to the contents of the documentation.

 Iron_Captain wrote:

We can tell whether some character is a number or a vowel (especially with Linear A, since Linear B which uses the same characters has been deciphered) with a fair degree of accuracy. For Linear A we can even spell out entire words and texts. Problem is those words are unintelligible because they are in an unknown language not closely related to any known language (Minoan is basically the Basque or the Hungarian of the Bronze Age I guess). The only thing we can somewhat guess at is place names (which are presumably derived from the Minoan language and therefore related). For example it may be that the Linear A word "Keniso" is the Minoan name for Knossos. But that is as far as we have gotten in the past 50 years, and as far as we are going to get until we find some bilingual text that could give us a starting point at finding out the meaning behind Linear A words. We can also guess which words are meant to be numbers and which are meant to be commodities, but the precise meaning still eludes us.

This makes it a pictoral code without a key, in codebreaking terms. Nothing especially unusual, except for the complete lack of double encryption. Agencies DO break codes like this. I am sure the Russians have a dab hand in it too.

 Iron_Captain wrote:

On a sidenote, I often do exercises like this where you get a text in an unknown language and then have to answer questions about it based on patterns (I even won a prize once in a competition). Sometimes you can even use patterns to translate an entire text even though it is an incredibly obscure language spoken only by some tribe deep in the Amazon forests. That is pretty cool. Thing is, you always need a starting point. To solve the equation you always need at least one known variable. With dead languages like Linear A, there is none. All variables are unknown which makes them completely impossible to solve or 'break'.

So you do this. Cool.
However maybe Ketara will come after you too.

I explained the dilemma earlier. You do need a starting point, your are correct, and to get one you need to make simultaneous breakthroughs because you have no starting point and have a cross match. Examples of how this was done was given in the previous comment.

Now your problem here is with the word impossible. The above problem is far less of an issue than double encryption, which should result in totally random code. But that is NOT unbreakable, just nearly unbreakable.
One advantage you have is with isolating where the tablets come from, similar to adding HumInt to the SigInt in the pattern. So for example if you have something that might be a trade document from a coastal city in asia minor you might look for references for the word for 'tin', because that is where this crucial import material will be arriving. Rural communities might not have that resource on their trade list. If the Egyptians, who we can read report of a saying of the hittites or a phrase they used in their documentation we could ook for that systematically and see if the same symbols in unrelated texts make sense.

A lot of this is translating the impossible into the possible via looking for the plausible. If you want an open unclassified example of this look at the history of Bletchley Park. While the early computers are the stars of the show, a lot of the codebreaking involved guessing words based on. So the first code of the day from St Nazaire was likely to be a weather report, because it gave an indicator of weather for central Europe all day. This was not always so, but it was often enough. So cross reference assuming codewords meant 'sunny' or 'storms' etc.

 Iron_Captain wrote:

Let's take my phrase again for example:

Maj jucha tivoj kajara jamäl.

And add a few more simple ones:

Maj jucha tivoj kajara obitsjimäl.
Jucha tivoj kajara jamita.
Maj jucha ja isakamäl jamidj.
Jucha ja obitsjimu.
Jucha tivoj kajara alisö obitsjimita.

Just from this small sample, you should be able to figure out which words are verbs, nouns and pronouns, and even what the word order of the language is (whether it is subject-object-verb or subject-verb-object etc.).

I personally cant tell one word from another. I am not a linguist. But I take your word for it.

 Iron_Captain wrote:

However, what you will not be able to figure out is what the sentences actually mean. Pattern recognition can only bring you understanding of the structure of a language. To understand what it really means, you need to have at least some translations.

Ok. looking at the above. Assuming it was part of a larger database of text fragments in a lost language. You could try and pattern match likely words depending on context of the document. It is appears to be a stack of poetry from a house. You also have a stack of what appears to be tax records from another part of the dig site. Say you hypothesis one word may mean cows, another may mean soldiers from your analysis of the tax documents. When reading the poetry as cross reference you hypotheses the sentence to say 'the cow owns five soldiers', this doesn't fit, there is a mistake somewhere, 'the soldier owns five cows' however makes sense.
You cant hope to cross reference this, but a computer can, and the larger the dataset the more cross references it can make.

Yes you will need to get a lucky break but when you do it can begin to unravel. Ideally you only need one tablet (preferably double sourced though) to have a list of things in numbered order, on the first year this happened, on the second that happened etc, and a computer might pick up the numerical sequence and thus decode a basic numbers system. You can work out a lot from even rough numbers. If a number is very large it might mean sheep or soldiers or bushels of grain, if a smaller number it might mean chariots or ships.

 Orlanth wrote:

 Iron_Captain wrote:

I see. But the thing is, all of this has already been done. By hand. Because we do not have a ten thousand clay tablets. The entire corpus of known Linear A texts for example fits on a few sheets of paper. And for many other undeciphered languages we have even less to go on..

If its done by hand it isnt done. People cannot cross reference every symbol and categorise it in relation to other surrounding symbols, it needs a computer.

As for the volume of data:

http://www.britishmuseum.org/about_us/departments/middle_east/facilities_and_services/study_room/studying_cuneiform_tablets.aspx

Linear A isn't cuneiform.

I never restricted to or even mentioned Linear A.

"Unknown languages on clay/stone tablets."

EDIT:- You know, I won't waste any more of anybody's time by continuing this discussion. Carry on gents. Feel free to delete this post, any regular mods.