What Is Information?

Faith & Science Conversation
8

Of course not because I don’t believe in your control freak god who has everybody’s life planned out ahead of time. I believe in a God who chose love and freedom over power and control, thus all living things can and do change to become what they choose to be because that is whole point for which God made life in the first place.

Another one of your additions to the text of the Bible?

I am an advocate of Sola Scriptura and that means people shouldn’t be rewriting the Bible to fit what they want to believe.

1 Like
9

Do you know the definition of taxonomy? I do not view God as a control freak. More a scientist with a bent towards efficiency. If a person is asked to name all the animals, why not classify them as well?

I think the issue is you see the time of Adam as a Disney movie of prehistoric fabrications. There is little indication of what life was like. There was agriculture of both plants and animals. There is a strong possibility they were more advanced than we are. They did have the best of the Spiritual and Physical aspects of reality. Not to be confused with history after the Flood when Humans had to start over in the stone age.

For the record I do not “have” a God. I just converse about God as recorded in the Bible whether that experience was good or bad. Leave the god making experts to other humans with more of an imagination. Reality is not what we make. Reality is what we experience. Free will is the choices we are allowed to make, regardless of the consequences. Consequences are the result of the universe running in accordance to the rules the universe was created with. If you think that rules are too deterministic, imagine how fun life would be without them. At least imagine for a split second because that may be all the time you get without rules.

10

You mean the idea that Noah had machine tools and advanced technology? Why not just go the whole way and claim that he was assisted by ancient aliens?

In actual fact, now that I mention it, Giorgio A Tsoukalos’s proposal that Noah’s Ark was a DNA bank is far, far, far more plausible than Answers in Genesis’s hyper-evolution…

4 Likes
11

Google: the branch of science concerned with classification, especially of organisms.

No the Bible says nothing about Adam being in charge of the classification of living organisms. All it says is that God brought animals to Adam and Adam named them and the way it says “that was their name” could be taken to mean that there was only one language at the time, especially in conjunction with Genesis 11 where it has God changing this to many languages.

That is another big difference between us then. A scientist investigates nature to find out how things work. But God already knows such things so calling Him a scientist would be nonsensical. I see God as a parent, who created the universe and life not as some kind of experiment but for an eternal parent child love relationship.

No. I see the way some people trivialize the story with excessive literalism as turning the story of Adam and Eve into a Disney movie for children.

There is no possibility they had anything like modern technology. They had no need of anything like that. And there is no indication that there was any loss of technology in the the story of Noah and the flood either. This is sounding more and more like you are rewriting the Bible as a science fiction fantasy novel. Nothing wrong with that really. The sci-fi Noah movie was cool. But that is just entertainment and I am not going to confuse either with reality and consistency with the objective evidence goes a long way towards making the story real.

I think the rules are a necessity for the existence of the physical phenomenon of life. I do imagine an existence without such rules after we die. And I think this explains part of the reason why we confront a divergence in ultimate human destiny known as heaven and hell. The other part being that only God has what is needed to make an eternal existence worthwhile.

1 Like
12

I take it then that you are referring to “Shannon Information” in which H(S) measures the optimal compression of the source messages. However this is actually not information but the entropy of the source message, S. As such, it is information ABOUT the message but it is not a measure of the semantic information IN the message. In fact the message need not have any semantic meaning.
Ref

“Frequently the messages have meaning; that is they refer to or are correlated according to some system with certain physical or conceptual entities. These semantic aspects of communication are irrelevant to the engineering problem. The significant aspect is that the actual message is one selected from a set of possible messages.” (Shannon 1948) [ref p6]

Shannon’s theory, taken by itself, is purely quantitative: it ignores any issue related to informational content.

When I was doing database programming we were asked what is the difference between data and information. Briefly “While data is an unsystematic fact or detail about something, information is a systematic and filtered form of data”. [More]

So the number of bits and bytes a file occupies when it is compressed is a measure of the size of the file. The file can contain data of variable quality, right down to completely random noise. From this data information can be extracted. However semantic items, such as meaning, reference or representation, are not amenable of quantification. [Ref p7] (Although in some cases it can be quantified.)

So your statement that " Insertion and duplication, by definition, add new information for starters." actually means that it increases Shannon Entropy but it will only rarely increase information. Mostly, as in my example, using only insertions and duplications, " Inserection and duplicplicplicplication, by definition, add new disinformation for starters." the entropy may have increased but the semantic information has decreased.

You are right that semantic information is highly context dependent, or highly specified, as in the case of your password, which would be complex specified information, and any insertion or duplication would reduce its information content to zero.

13

Yes Chris, it is Shannon information that I’m talking about when I talk about “information.” Good to see you’re on the ball there.

There was a thread here a couple of years ago (exactly!) that had a much more rigorous treatment of this subject. You may want to take a look at it:

You’ll need to read through it (at least the original post) to grasp it properly, but the basic gist of it is that the claim that mutations can not add new information – in whatever sense you choose to define information – is either unproven, unprovable, or wrong.

14

It will take a while. Just the first post is 5 pages when I copy it into my word processor and I’m not going to read the remaining 94 posts. From my first quick read I’m going to stick my neck out and say that @Swamidass is wrong. More after I’ve had time to digest it.

15

@jammycakes, @Swamidass

Swamidass has made a basic confusion about the meaning of information, and I say this knowing that his “PhD is specifically in “Information and Computer Science”, with emphasis (in my case) on information.”

In the second paragraph of Shannon’s paper of 1948 he says "The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point. Frequently the messages have meaning; that is they refer to or are correlated according to some system with certain physical or conceptual entities. These semantic aspects of communication are irrelevant to the engineering problem."

The information Shannon is referring to is the information required to transmit a message but not the information in the message; as Shannon says that is irrelevant to the engineering problem.

While the amount of information required to transmit a message is often called Shannon Information it is actually a measure of the Entropy, so it should be called Shannon Entropy. Unfortunately this misnomer encourages some people to conflate the calculated entropy of the message with the semantic information within the message. This is similar to conflating the weight of a box with the contents of the box. For shipping purposes the weight is the relevant measure and it doesn’t matter if the box contains diamonds or dirt.

But once you conflate the entropy OF the message with the information IN the message then it is an easy step to conclude that maximising the entropy maximises the information contained in the message; and hence conclude that information is maximised when the message is completely random.

Consider Abraham Lincoln’s famous Gettysburg Address (1473 characters)

Four score and seven years ago … and that government of the people, by the people, for the people, shall not perish from the earth.

Replace it with the following and you have reduced the Shannon Entropy drastically but lost the information contained in the message.

111111111111111111111111111111 … 11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111

Replace it with the following and the Shannon Entropy will be high but again the information in the message is lost

1rdn74mao.28spawjjk16wpstedyw0 … 12bd03payentwpd674ba;m81$%(dgepsn52plFQ2mduw06%dkwatpeyW710FtD&1brghepWQL$v%1hsdeQyw72 qrT2H84s-

16

No Chris, @Swamidass does not conflate Shannon information (entropy) with semantic information. In fact he clearly differentiates between the two when he talks about information content and mutual or shared information:

NOTE TO THE CONFUSED: This discussion can get confusing because of the many definitions of “information” in common speech, and also that there are two main types of information that work in different ways: (1) entropy or information content and (2) mutual information or the shared information. Information content is the amount of information in a single entity (and is measured as the entropy). Mutual information is the amount of information shared by multiple entities (and is a measure of commonality, and is equal to the difference between two entropies). When communicating with the public, it is hard to keep these two different types of information straight without devolving into dense technical language. But if you detect a contradiction in what I wrote, this is probably the reason. Because these two types of information behave similarly in some cases, and exactly opposite in others.

The important point that he makes is that semantic information is (a) poorly defined, and (b) impossible to quantify (since it is impossible to rule out the fact that there may be a meaning in a seemingly random string of characters that we had not managed to identify and decode). It is this specific point that you need to address, because it is this specific point that tells us that it is simply not possible to claim, on information theoretic grounds alone, that mutations can not produce new semantic information.

5 Likes
17

I think I actually covered this In my previous post but I will risk repeating myself.

Claude Shannon in 1948 was addressing a technical problem in communications and was using the word information in that context and within the conventions of that time. He was not trying to measure information itself but only the size of the message that needed to be sent via the communications system. He said specifically that "These semantic aspects of communication are irrelevant to the engineering problem.

In his paper Shannon referred specifically to “7. THE ENTROPY OF AN INFORMATION SOURCE” but unfortunately over time this has come to be called Shannon Information, a misnomer.

Consider the analogy of sending a parcel by a courier company that charges by volume, the size of the parcel. They measure the length, width, and depth and calculate the volume. The calculated volume provides information about the size of the parcel you are sending, but volume is not the parcel nor the contents of the parcel. Neither is volume in general information about the parcel but only the specific calculated value.

Similarly Shannon Entropy is a measure of the size of the message. It provides information about the message but it is not the message nor is it the information within the message. Hence it is invalid to equate Shannon Entropy with Information as Swamidass does in his post.

Now if you want to talk about what information actually is that is a whole other subject and beyond my pay grade. I did find Werner Gitt’s book “In the Beginning Was Information” helpful in in this regard.

Unfortunately @Swamidass has advised me that he is not able to respond in this forum.

[edit]
The message itself is not information either. The information is coded in the message. I might start with the thought about what we will have for dinner tonight. That thought is coded into English words which are represented by strings of characters which make the message, “Bangers and mash for dinner tonight”. My wife gets the message and decodes the letters to words to information. She might then reply “What, again?”

The encoding and decoding steps can go wrong if the sender and receiver use different coding systems; like when I downloaded some product information and found it was in German. The information had been encoded into the message and transmitted successfully but I didn’t have the correct system to decode it back into information. I had this problem in Germany last year when trying to order a gluten free meal in a restaurant. I could encode the request into English, transmit it successfully, the hearer received (heard) the message but could not decode it. In the reverse direction I could receive but not decode their messages. (We ended up with a meal of spargel.)

A message can be encrypted and it will then appear as gibberish or random data because we don’t have the correct decoding system. This is not the case with random noise which does not have information encoded by any system. This is another reason why Swamidass is wrong when he says " Another surprising result is that the highest information content entity is noise , exactly the opposite of our intuition of what “information” actually is."

18

Yes Chris, you are repeating yourself. You are also repeating points that @swamidass made in his post. And the points you are repeating do not address the question at hand.

There are two different meanings of the word “information” under discussion here.

  1. How @swamidass uses it (to refer to Shannon entropy)
  2. How you insist it should be used.

The point I made is that @swamidass acknowledged your definition. He referred to it in his post as “shared information.” He also said that it does not have a rigorous mathematical definition, and can not be properly quantified. And however you insist that the word “information” should be used, a rigorous definition is essential before you can insist that mutations can not introduce more of it.

Nothing you have said addresses this specific point.

I’m not ruling out the possibility that there may be an information theoretic argument against undirected evolution, but if there is one, it will be complex, difficult to understand, and even harder to fact-check. And even if such a proof is discovered, it won’t change the fact that life in the Cretaceous was very different from what it is today; it won’t change the fact that humans and animals have the appearance of being related; and it certainly won’t reduce the age of the earth to six thousand years.

1 Like
19

At the risk of jumping into this without having read all leading up to it … I think I know what Swamidass (and others before him) mean when they refer to noise as having the highest of information content. Think of it this way. You can probably gs wt I’m sag here evn thgh mny of the Let*s are msnisg or scrbelamd. And the reason you could probably read most of that sentence is that it was predictable enough for your brain to fill in or correct the missing parts. In other words, letter-for-letter, due to redundancy, the characters are not each carrying a lot of information weight and the message can still come through with a good many of them missing or out of order. Contrast that with me sharing a phone number with you: 271-555-8?42. In this case each and every character is critical and there is no redundancy or any way for your brain to fill in or correct even just one digit since each digit is independent of all others around it. You have no basis for guessing which of the possible 10 digits the ‘?’ should be. This latter string of information is “noise” in the sense that every bit of information is independent of the bits that surround it. I believe that is what is meant when ‘noise’ is referred to as carrying, letter-for-letter the most ‘information’, even if it is “truly random” (whatever that is - and if it even exists). All it would mean is that it is, as you say, indecipherable to everybody as there would be nothing there to decipher. So your point is well-taken that it certainly isn’t information in the common sense way we think of it. But distinguishing those situations would in practice often be impossible for us. 59265358979323846 will be noise to one person, but a segment of the digits of pi for those who can recognize it as such. And if such a sequence was turned into an electrical signal sent to speakers you would hear something indistinguishable from white noise (static) - precisely because of its high information content density, whereas if you send the digitized version of the decimal for 1/3 (33333333…) then you will hear … nothing. Because there is almost no information added in the sequence by any particular ‘3’. Remove or rearrange as many of them as you want and the sequence suffers no loss whatsoever.

3 Likes
20

A nice reference to pi on pi day.

1 Like
21

Just for the record, the proper pronunciation of π is “pee,” and I have commemorated the day accordingly.

3 Likes
22

I think you’re right about what Swanidass et al. are saying and they’re wrong.
The first point is that there is no such thing as Shannon Information. If you go back to Shannon’s 1948 paper he did not define the information in the message but the Entropy of the message. Shannon was dealing with the specific technical problem of how to faithfully transmit a message from sender to receiver. The Shannon Entropy provides useful information ABOUT the message but it nothing to do with information IN the message, something Shannon acknowledged in his paper.

If you look at my example above of the Gettysburg Address;

; you will see how overwriting it with random noise increases the Shannon Entropy but destroys the information in the message.

In fact this is what you do to safely delete a computer file; overwrite it with random data (several times) so that the information in the file is destroyed. The file can still be read but the information in it is destroyed.
(I just though of this example, it’s one IT professionals could probably relate to.)

23

This is not quite right. Originally files were deleted by removing the information that pointed to where they were stored on the disk. It was possible to still read the information in the deleted file if you could find where it was on the disk. To prevent this the area of the disk used for a file is overwritten with any data, all 0’s or 1’s actually works. I have no idea how this is supposed to apply to this discussion. Replacing information with other information is what is actually happening in your example.

From a computer science perspective Shannon’s information entropy tells you the minimum bandwidth needed to transmit a signal with a given maximum frequency. It is also related to how much a file can be compressed and still recovered completely. Both of these are examples of information from my perspective at least.

1 Like
24

What Does File Shredding Do?

To shred a file, you run it through a program that overwrites it several times with other data. It doesn’t actually get “shredded” in the sense that paper documents do.

To use that analogy, it’s more like you’re taking a paper document, erasing all the words, and writing over the top of them with a bunch of nonsensical words. And just like erasing the word off a page will leave a trace behind, it’s technically possible that overwriting digital data will too. So you do it again and again until you can no longer see the original data underneath.

So tell me Bill, what IS the information in all 0’s or 1’s, or in completely random data. All 0’s or 1’s will give low entropy but will have no meaning, semantic information. Random data will have high entropy but will have no more semantic information than all 0’s. It will be meaningless. On the other hand all the information in the original file is gone. Can you show that the overwriting data has more information instead of just showing it has more entropy?

25

You can only define meaning when you know what the information represents. A file containing all 1’s could represent a graphics file that only contains white while the file with all 0’s would represent black. Those files convey information even given the very low entropy. A file with completely random data can also represent information if it contains a high resolution photographic image. That is why I said you are only changing the information in your example.

I didn’t say it has more information. You are just changing the information content of the sectors on the disk. Which brings us back to the point James was making.

Edit to add:

I should kick myself for forgetting my cardinal rule, always check the sources. And in looking at Shannon’s paper I found this just a couple of lines down from the quote you used.

Bolding by me of course.

26

If it contains a high resolution photographic image then the data is not random, even if it has high entropy.

If indeed the information the sender intended to transmit was a totally black or white image then that would be information.
However if the file was overwritten to destroy the information then it is only you projecting a meaning onto the contents. This is akin to finding a picture in the clouds or a word in a bowl of alphabet soup. In a random text file there will probably be an occasional string that can recognised as a word but again that is you imposing a meaning on that particular string.
However if a file “Lincoln-Gettysburg.txt” containing a copy of Lincoln’s Gettysburg Address is edited to contain a selection from Mein Kampf then it will contain information, albeit false information if the intention is to attribute those words to Lincoln.

First, this is not referring to the calculated value of entropy which is only defined later in the paper. Second, this is information about the message rather than the information in the message.

I will say again that Shannon was addressing “The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point.”

This actually misses both ends of the communication of information. When I have the thought about what we will have for dinner I express that thought in words and then represent those words as strings of characters. “Bangers and mash for dinner”. Only then does Shannon take over to ensure the accurate transmission of that message. We do this so naturally we can don’t even notice what we are doing. But you can see that it is coding of information because a German would produce something completely different to convey the same information, as would a Chinese.

Also Shannon is concerned with transmitting this message through an electrical communications system. I could simply speak. Abraham Lincoln’s address was delivered verbally to multiple receivers simultaneously.

Then the receiver must perform this process in reverse to get the information contained in the message. The sender and receiver must use the same coding conventions to encode and decode the message. I cannot decode a message sent to me in Chinese. Similarly you can’t sensibly open a text file in a graphics program; you might get something but it won’t be the information the sender put in the message. This is why it is nonsensical to say that a file that originally contained the Gettysburg Address could, after being overwritten with 1’s, represent a white graphics image.

27

The quote is from the Introduction where Shannon is discussing the purpose of the paper. The number he is talking about will be defined as entropy later in the paper.

All through the paper Shannon is talking about information. In fact he defines information as the selection of one message from the set of possible messages. He isn’t concerned with meaning but with information. Which is why people refer to information as they do even if it isn’t the way you want to define it.

1 Like