“Improved Synonym Approach to Linguistic Steganography” Design and Proof-of-Concept Implementation Aniket M. Nanhe Mayuresh P. Kunjir Sumedh V. Sakdeo B.Tech Comp. Sci., College of Engineering, Pune, India. B.Tech Comp. Sci., College of Engineering, Pune, India. B.Tech Comp. Sci., College of Engineering, Pune, India. nanhe.aniket@gmail.com mayuresh.kunjir@gmail.co m sumedhsakdeo@gmail.com Abstract This paper develops a linguistically robust Linguistic steganography approach using synonym replacement, which converts a message into semantically innocuous text. Drawing upon linguistic criteria, this approach uses word replacement, with substitution classes based on traditional word replacement features (syntactic categories and subcategories), as well as features under-exploited in earlier works: semantic criteria, inflectional class, and frequency statistics. The original message is hidden through use of a cover text which is shared between sender and receiver. This paper also presents a new approach of sharing the cover text and changing it periodically to make the algorithm safe from steganalysis. 1. Introduction While current encryption techniques are sufficiently advanced to make code-breaking practically impossible, one major drawback of current encryption methods is the ease in identifying an encrypted text—they do not resemble natural text in any way. Steganography attempts to answer this need, acting to conceal the message's existence, in order to transmit encrypted messages without arousing suspicion. Steganography is the art and science of concealing a secret message inside a cover object. When the secret message is in digital form, it leaves enormous choices for the cover objects. For instance, one could hide digital information inside images, audios, binaries, videos, texts etc. Steganography can be classified depending upon the type of cover objects used. These cover objects could be images, audio, binaries and as in our case, natural language text. The study of this subject in scientific literature dates back to 1983, when Simmons formulated it as ―The Prisoners‘ Problem‖. It says, ―Alice and Bob are in jail and wish to hatch an escape plan; their entire communication pass through the warden Wendy; and if Wendy detects any suspicious messages, he will frustrate their plan by throwing them in solitary confinement. So they must find some way of hiding their secret message in an innocuous looking cover text‖. Information hiding has taken one form in image based steganography, utilizing minimal changes in pixels or watermarking techniques. While text-based messages have also been used within image-based maneuvers, by modifying the white space between letters and by minutely changing the fonts, this has proved less fruitful because text can be retyped and is often altered in the conversion from one program version or platform to another. Proving more productive, as well as resistant to the difficulties surrounding the re-typing of text-based messages is lexical steganography, which uses linguistic structures to disguise encryption of text messages such that the appearance of the message remains semantically and syntactically innocent. This paper presents a new approach of Linguistic Steganography using synonym replacement, a linguistically-informed alternative to existing text-based steganography systems. This approach adds extra features like inflectional class and frequency statistics thereby producing semantically and syntactically correct text which is more natural in appearance to human eye. There is one more area which is under-exploited in earlier works: sharing of cover text and changing it periodically. The cover text is very critical as it is the text which is transformed and sent over communication channel. Our aim is to make steganalysis difficult by altering cover text frequently so no suspicion would be detected. This paper discusses a new approach of achieving this by exploiting the part of cover text which remains unchanged in transformation. This paper analyzes Past Research in Section 2, Basic Algorithm in Section 3 and Cover Text Selection in Section 4. 2. Past Research Lexical steganography has had three main veins of research: watermarking techniques that manipulate sentences through syntactic transformations a.k.a. ontological techniques, word replacement systems both with and without cover texts, and context-free grammars such as NICETEXT. We will see the work carried out in each of these techniques. 2.1. Syntactical Steganography The approaches to syntactical steganography exploit the syntactic structures of a text. The approaches make use of Context Free Grammars (CFG) to build syntactically correct sentences. The famous algorithm of CFG based Mimicry developed by Peter Wayner[1] comes under this category. There is another famous algorithm by Chapman et al.[8], NICETEXT, also based on CFG. NICETEXT uses the cover text simply as a source of syntactic patterns: by running the cover text through a part-of-speech tagger, NICETEXT obtains a set of "sentence frames," e.g. [(noun) (verb) (prep) (det) (noun)] for ‗I sat in the tree.‘ It also compiles a lexicon of words found in the cover text via part-of-speech tags, with each word in the lexicon associated (arbitrarily) to either of the binary digits 0 or 1. In encryption, the plain text message is converted into a sequence of binary digits. A random sentence frame is chosen and the part of speech tags in it are replaced by words in the lexicon according to the sequence of binary digits. Although, NICETEXT produces syntactically correct sentences; it fails on the count of semantics. The output text is almost always set of ungrammatical and semantically anomalous sentences. Another factor worth considering is the density of encryption within the cover text. Ideally, the cover text should work to hide the word frequencies and syntactic structure of the hidden plain text message. Steganographic goals encourage sparse encryption, which does not alter a majority of the text by the word replacement. NICETEXT encryption is maximally dense—every word within the final encrypted cover text is conveying hidden information. Given that each encrypted word is part of the original information bearing message and common word usage patterns are unavoidable, this is problematic for the original steganographic intent: avoiding detection and producing naturalistic text. In general, syntactical steganography techniques produce text having syntactically wellformedness without semantically wellformedness. It can be seen from Chomsky‘s famous sentence ―Colorless green ideas sleep furiously‖. 2.2. Lexical Steganography In lexical Steganography lexical units of natural language text such as words are used to hide secret bits. The most straightforward subliminal channel in natural language is probably the choice of words. A word could be replaced by its synonym and the choice of word to be chosen from the list of synonyms would depend upon secret bits. For example consider a sentence – ―Pune is a nice little city‖ Now, suppose list of synonyms for nice is {nice, wonderful, great, and decent}. Each of the synonyms can be represented by two bits as shown in the table: Word Code Nice 00 Wonderful 01 Great 10 Decent 11 Table 1: Lexical code table Depending upon the input secret bits appropriate synonym for ‗nice‘ will be selected and put in the stego text. So, the possible stego texts could be: a) b) c) d) Pune is a nice little city. Pune is a wonderful little city. Pune is a decent little city. Pune is a great little city. The lexical techniques produce better quality text than syntactical techniques. It is hard to find presence of hidden message for statistical attacks. The replacements are critical part of these techniques. To give an example, in the above mentioned synonym replacement approach, some words can have more than one sense. (Noun ―bank‖ has two senses – ―a long pile or heap‖ or ―an institution for receiving, lending, and safeguarding money and transacting other financial business‖) If we don‘t use synonym having same sense as that of original word, the output will look suspicious. 1. Bring those instruments. 2. Bring those tool. A further impediment to synonym based word replacement is inflection classes (i.e., legal and illegal word combinations). (2) replaces a plural noun ‗instruments‘ of (1) by its singular synonym noun ‗tool‘; thus making sentence grammatically incorrect. 2.3. Ontological Technique Of the techniques considered herein, the ontological one is the most sophisticated approach with respect to modeling semantics. Instead of implicitly leaving semantics intact by replacing only synonymous words while embedding information into an innocuous text, an explicit model for ―meaning‖ is used to evaluate equivalence between texts. Atallah et al.[4] watermark texts by manipulating and exploiting the syntax (formal word order and grammatical voice) of sentences. Through common generative transformations (clefting (4), adjunct fronting (5), passivization (6), adverbial insertion (7)), the syntax of each sentence is altered: 3. The lion ate the food yesterday.(original sentence) 4. It was the lion that ate the food yesterday. 5. Yesterday, the lion ate the food. 6. The food was eaten by the lion yesterday. 7. Surprisingly, the lion ate the food yesterday. The Ontological techniques though have some problems. The transformations sometimes affect the semantics of a text. Newer theories of language argue for the interconnectedness of the semantic and syntactic levels, demonstrating that the syntactic pattern is itself inherently meaningful. Furthermore, statistically, various syntactic structures (word orders) are not equal in distribution: different genres of text have wildly different syntactic structures, and replacing such structures freely could create a text which is trivially broken by statistical methods—a security threat to the program. 3. Basic Algorithm The algorithm replaces all the nouns, adjectives, verbs and adverbs of cover text by their respective synonyms. A semantically and orthographically correct text is used as cover text to hide messages. We use a word dictionary to get synonym. The input text to be hidden is compressed using Huffman Compression Algorithm and a string of bits is generated. The input bits are consumed in selection of synonyms. The algorithm works in stages. The various stages of the algorithms are: 3.1. Part-of-speech Tagging The basic requirement of this algorithm is, a cover text should be shared between sender and receiver. Natural Language Processing is done on the cover text in order to determine the part of speech of each word. This is essential part of the algorithm as we are going to replace only common nouns, adjectives, adverbs and verbs in the cover text. A Parts-of-speech tagger is applied on cover text which outputs each word followed by its part-ofspeech. 3.2. Input Compression The input secret text is treated as binary bit string. These bits are to be used in synonym replacement stage to make a choice of synonym that is to be used in place of a word. Using standard ASCII representation, we need 7 bits for each character of input. So we need to hide (7 * ‗number of characters‘) bits. We can improve on this number by exploiting characteristics of English language. Some characters appear more frequently in normal English text than other characters. If we use less number of bits for such characters, we can easily reduce number of input bits to be hidden. To achieve this, we use Huffman Compression algorithm. On an average, Huffman coding reduces the size of input bit string to 33% of the original. 3.3. Synonym Replacement This stage is the core of the algorithm. The actual task of hiding bits into a cover text is carried out here. The inputs to this stage are the compressed bit string and tagged cover text at sender‘s side and receiver needs encoded text (a.k.a. stego text) and tagged cover text. This stage makes use of dictionary to find replacements for word. 3.3.1 Dictionaries We use three dictionaries here: a. WordNet2.1 English dictionary WordNet is an open source English dictionary containing almost all English words. We can get all synonyms of a word using this dictionary. A word may have more than one sense in which it can be used. WordNet provides output in terms of ‗synsets‘. A synset defines a sense of a word. Each synset contains all the synonyms of given word which can be used in that particular sense. e.g.: Synonym Sets of the word ―travelling‖ are: 1. travel, go, move, locomote 2. travel, journey 3. travel, trip, jaunt 4. travel, journey WordNet also provides frequency of occurrence of a word in normal English text. This information is very useful in our algorithm using which we can encode the synonyms. Huffman coding is used here again so that more frequently occurring synonyms get shorter codes and vice versa. This is important from Word Sense Disambiguation aspect, wherein the WordNet‘s 1st Sense assigns some frequency to each synonym of the word. This frequency is assigned to a word depending upon the use of that synonym for that particular word in normal English text. Assigning a shorter code to most frequently used synonym ensures maintaining proper word sense. ―travelling―, the stego text should contain present participle for of the ―go― to maintain the tense of the sentence. So verb dictionary provides this inflected form of the base form ―go‖ for actual replacement. c. Noun Inflection dictionary A noun can be either in singular or plural form. Again as the case with verbs, WordNet always gives the synonyms of a noun in their singular form. If we replace a plural by its singular synonym noun, we will get grammatically incorrect sentence. So we use separate noun dictionary to avoid this situation. We maintain a list of nouns (about 89,051 nouns) in their singular as well as plural forms. Before replacing a noun by its synonym, we check whether both are in same form. If not, we select appropriate form of synonym noun from noun dictionary and replace original by it. e.g.: Tagged Cover Text: A/DT group/NN of/IN frogs/NNS were/VBD traveling/VBG through/IN the/DT woods/NNS ,/, The noun ―frogs‖ is plural form of the word frog. Suppose synonym Gaul is selected by input bit string; we need to ensure that it is plural as ‗frogs‘ is a plural. So we use this dictionary to obtain singular and plural forms of the nouns present in dictionary. b. Verb Inflection dictionary 3.4. Synonym Replacement A verb can have many inflected forms like present participle form, past tense form, past participle form and base form. When we try to find synonyms for a verb, WordNet always gives the synonyms of a verb in their base form irrespective of inflected form of input verb. If we replace a verb by a synonym in its base form, it will make output sentence grammatically incorrect. To avoid this situation, we use a separate verb dictionary. We maintain a list of all verbs (about 16,064 verbs) along with their all inflected forms in a separate file. Before replacing a verb by its synonym, we check whether inflected forms of both original verb and its synonym match. If they don‘t match, we select appropriate inflected form of synonym verb from verb dictionary and replace original by it. Figure 1 shows the mechanism that is carried out at sender end. As can be seen, tagged text obtained from stage 1 is scanned. Whenever a noun, adjective, verb or adverb is found, its synonyms are obtained from WordNet. All synonyms are put in a frequency table; the frequencies are obtained from WordNet. Huffman coding is done on this frequency table to obtain codes for all synonyms. By using frequencies, we achieve word sense disambiguation also, as more frequently used senses get shorter codes so that they have higher probability of being used. After building the encode table, we use input bit string to select one of the synonyms from the table. If we are replacing a verb, the inflected forms are checked and appropriate form of verb is obtained from verb dictionary. Similarly if we are replacing a noun, the singular or plural form is selected from noun dictionary in accordance of original noun‘s form. Otherwise the selected synonym is put in place of original word. Appendix shows examples of sample stego text generated from cover text by hiding the secret text. e.g.: Tagged Cover Text: A/DT group/NN of/IN frogs/NNS were/VBD traveling/VBG through/IN the/DT woods/NNS ,/, Suppose, synonym ―go‖ is selected for replacement from WordNet as a synonym of bits are obtained and these are appended to output string. The output string when decompressed, produce original secret text. 4. Cover Text Selection Figure 1: Sender end Steganalysis is identifying existence of a secret message. This is obvious as the field of steganography aims to conceal the existence of a message, not scramble it. Our approach uses Word Replacement in cover text. As only few words are replaced by their synonyms majority of text remains unchanged. If same cover text is used again and again, an attack ―Known Stego-Text‖, in which intruder keeps a track of text being sent on the communication medium is possible. To prevent text from steganalysis, the cover text needs to be changed periodically. Our approach uses a book, a collection of different chapters. Book should be privately owned by sender and receiver. One of the chapters from the book can be selected as cover text. The choice of the chapter is randomly decided by sender. For reverse transformation at the receiver end, same chapter should be selected as cover text. To achieve this, we exploit the unchanged part of the cover text. Receiver decides which chapter is to be used as cover text from stego text. This approach calculates difference between individual chapters in book and stego text and selects the chapter with minimum difference. Initially, each chapter in book is scanned once to obtain a code for each sentence. Words which are common nouns, adjectives, adverbs or verbs are ignored from the calculation of the code. Values of other words are calculated using ASCII values of characters and position of the word in that particular sentence. All these values are summed up to determine the code for that sentence. Similarly, codes for all sentences are calculated. e.g.: ―This is important for me‖. Figure 2: Receiver End Reverse algorithm is carried out at receiver end. Figure 2 shows the mechanism. Tagged text is scanned for noun, adjectives, verbs and adverbs. When one is found, its synonyms are obtained from WordNet. As done at sender end, frequency table and later encode table is formed using frequencies of the words. At the same time, stego text is also scanned. From that stego text, we obtain the synonym selected at sender‘s side. Then from the table, its Code =‗t‘ * 1 + ‗h‘ * 1 + ‗i‘ * 1 +‗s‘ * 1 + ‗i‘ * 2 + ‗s‘ * 2 + ‗f‘ * 4 + ‗o‘ * 4 + ‗r‘ * 4 + ‗m‘ * 5 + ‗e‘ * 5 ―important‖ is not used in calculation of the code for this sentence as it is an adjective. Similar algorithm for code generation is applied on stego text. Code of stego text is compared with codes for all chapters. Ideally codes for all sentences of stego text should match with codes of a chapter which was used as cover text by sender. But our algorithm allows compound word replacements (―travel‖ can be replaced by ―move around‖). This causes a difference between codes of stego text and chapter to be used as cover text. This difference although is very small compared to difference for other chapters. So the chapter with minimum difference is selected as cover text. 5. Conclusion The field of Linguistic Steganography is very interesting as it conceals the very existence of secret message from intruder, which is not achievable by cryptography. The Synonym replacement approach used for Linguistic Steganography produces innocuous looking English text thereby making detection of secret message very hard. The famous Stego-Turing Test states that it is very hard for a computer to alter a natural language text in a way that is undetectable to a human. Many approaches have been carried out in the past in doing this. But none has been able to solve the problem. Though our solution doesn‘t solve the problem, it produces better quality of output than previously done approaches. Also we give a new approach to dynamically choose a cover text from chunk of text being shared by sender and receiver. This allows user to use different cover text for hiding message each time, thus making steganalysis difficult. Synonym Replacement Approach to Linguistic Steganography using Inflection classes, Frequency Statistics and Dynamic Cover Text Selection is a new improvement in the field of Linguistic Steganography and provides a very good, efficient tool for Information Hiding. 6. References [1] P. Wayner, ―Mimic functions,‖ Cryptologia XVI, pp. 193–214, July 1992. [2] P. Wayner, ―Strong theoretical steganography,‖ Cryptologia XIX, pp. 285–299, July 1995. [3] P. Wayner, ―Disappearing CryptographyInformation Hiding: Steganography & Watermarking‖, 2nd edition Morgan Kaufmann Publishers, Los Altos, CA 94022, USA, second ed., 2002 pp. 67-126, pp. 303-314. [4] M. J. Atallah, V. Raskin, M. Crogan, C. Hempelmann, F. Kerschbaum, D. Mohamed, and S. Naik, ―Natural language watermarking: Design, analysis, and a proof-of-concept implementation,‖ in Information Hiding: Fourth International Workshop, I. S. Moskowitz, ed., Lecture Notes in Computer Science 2137, pp. 185–199, Springer, April 2001. [5] M. J. Atallah, V. Raskin, C. F. Hempelmann, M. Karahan, R. Sion, U. Topkara, and K. E. Triezenberg, ―Natural language watermarking and tamperproofing,‖ in Information Hiding: Fifth International Workshop, F. A. P. Petitcolas, ed., Lecture Notes in Computer Science 2578, pp. 196–212, Springer, October 2002. [6] K. Bennett, ―Linguistic steganography: Survey, analysis, and robustness concerns for hiding information in text,‖ Tech. Rep. TR 2004-13, Purdue CERIAS, May 2004. [7] M. T. Chapman, ―Hiding the hidden: A software system for concealing ciphertext as innocuous text,‖ Master‘s thesis, University of WisconsinMilwaukee, May 1997. [8] M. T. Chapman and G. I. Davida, ―Hiding the hidden: A software system for concealing ciphertext as innocuous text,‖ in Information and Communications Security: First International Conference, O. S. Q. Yongfei Han Tatsuaki, ed., Lecture Notes in Computer Science 1334, Springer, August 1997. [9] M. T. Chapman, G. I. Davida, and M. Rennhard, ―A practical and effective approach to largescale automated linguistic steganography,‖ in Information Security: Fourth International Conference, G. I. Davida and Y. Frankel, eds., Lecture Notes in Computer Science 2200, p. 156ff, Springer, October 2001. [10] R. Bergmair, ―Towards linguistic steganography: A systematic investigation of approaches, systems, and issues.‖ final year thesis, April 2004. handed in in partial fulfillment of the degree requirements for the degree ―B.Sc. (Hons.) in Computer Studies‖ to the University of Derby. [11] I. A. Bolshakov, ―A method of linguistic steganography based on collocationally-verified synonymy.,‖ in Information Hiding: 6th International Workshop, J. J. Fridrich, ed., Lecture Notes in Computer Science 3200, pp. 180–191, Springer, May 2004. [12] K. Winstein, ―Lexical steganography through adaptive modulation of the word choice hash,‖ January 1999. Was disseminated during secondary education at the Illinois Mathematics and Science Academy. The paper won the third prize in the 2000 Intel Science Talent Search. [13] A. J. Tenenbaum, ―Linguistic steganography: Passing covert data using text-based mimicry.‖ final year thesis, April 2002. Submitted in partial fulfillment of the requirements for the degree of ―Bachelor of Applied Science‖ to the University of Toronto. [14] Vineeta Chand and C. Orhan Orgun, ―Exploiting linguistic features in Lexical Steganography‖, Proceedings on 39th Hawaii International Conference on System Sciences - 2006. APPENDIX Secret Text: Escape from jail today evening. Sample Cover Text: EVER since I have been scrutinizing political events, I have taken a tremendous interest in propagandist activity. I saw that the Socialist-Marxist organizations mastered and applied this instrument with astounding skill. And I soon realized that the correct use of propaganda is a true art which has remained practically unknown to the bourgeois parties. Only the Christian-Social movement, especially in Lueger's time, achieved a certain virtuosity on this instrument, to which it owed many of its successes. But it was not until the War that it became evident what immense results could be obtained by a correct application of propaganda. Here again, unfortunately, all our studying had to be done on the enemy side, for the activity on our side was modest, to say the least. The total miscarriage of the German 'enlightenment ' service stared every soldier in the face, and this spurred me to take up the question of propaganda even more deeply than before. There was often more than enough time for thinking, and the enemy offered practical instruction which, to our sorrow, was only too good. Sample StegoText: ever since I have been scrutinizing political cases, I have taken a tremendous interest in propagandist action. I experienced that the SocialistMarxist organizations subdued and practiced this tool with astounding accomplishment. And I shortly recognized that the right use of propaganda is a truthful art which has stayed much unknown to the businessperson parties. merely the Christian-Social move, particularly in Lueger 's time, accomplished a sure virtuosity on this instrument, to which it owed many of its successes. But it was not until the War that it turned evident what immense effects could be found by a right application of propaganda. hither once more, unfortunately, all our considering had to be made on the enemy side, for the action on our side was modest, to say the least. The total stillbirth of the German ` Nirvana ' service starred every soldier in the face, and this spurred me to bring up the inquiry of propaganda even more deeply than ahead. There was frequently more than plenty sentence for mentation, and the opposition offered practical education which, to our regret, was only too good.