Modeling Geostrategic Dynamics for the Indian Subcontinent

[BajKhedawal]

I had posted this elsewhere and Ramana gave a few pointers, and redirected me here as well. Please bear in mind compared to heavyweights here; I am like a nanha mujahid writing a 5th grade essay. So I don’t have access to lot of things. But don’t let that hold back your recommendations; I will work with what I can get.

A Request (I don't know where else to put this for better visibility)

I want to do an entry level data mining project on state sponsored terrorism (germinating from in and around papistan), but need guidance on where to get academically ACCEPTABLE data.

Data mining works well when you're searching for a well defined profile, pattern, and a reasonable number of attacks per year. But how do you measure indefinable patterns in terrorism?

Some possible examples are:

Phone data
Social network data
Financial transactions
Wire transfers
Border crossings
Public records
Travel records
Illicit drug/arms trade

Again all of the above boils down to academically ACCEPTABLE data, an example

http://www.justice.gov/dea/pubs/cngrtes ... _text.html

Chart compares Opium Production in Afghanistan and Pakistan (respectively) for years 1993 to 2000 as follows
(metric tons):

I need help from the stalwarts here, else I will have to do benign stream flow analysis and drought project. Alternate ideas backed with raw (pun or no pun ?) data are welcome

Thanks in advance

[brihaspati]

^^^ Are you looking for statistical models for "predicting" terror parameters given measurable or observable relevant variables like drug/arms trade value/volume etc? If so, one commonly used predictor variable is the number of casualties. More sophisticated analysis uses "type" of terror outrage, actual hardware used/modus operandi, and financial impact. Valuation (assigning quantitative values) to the latter is of course much more difficult and hotly contested.

Application of utility theory to assigning values depends on actual interviews with "terrorists" partly. It can be more easily obtained for non-terrorist participants/victims in a terror outrage. But here again there could be limitations. A gov or security forces or even businesses may not reveal their true utility either.

[RamaY]

BajKhedawal,

I have gathered some info before to prepare basic trends and annual totals for BRF posting. But can help you in data collection (looking for data sources etc.,). I will leave the analysis and inference part to you.

In return, I would like to have complete access to the database, a copy of it if you will. I would like to do my own research on it for BRF.

If it is a deal, pls write to me at "r a m a y dot b r f at g m a i l"

thanks

[brihaspati]

On the Indian subcontinent, droughts (in water terms - rainfall) are very highly correlated historically with "terror" attacks and invasions or raids. The data for this can be more easily obtained. But the area of analysis needs to include the "greater India" rim zone too - where the "terror" is sourced from. Rainfall data, and aridity data is easily obtained.

[RamaY]

On the Indian subcontinent, droughts (in water terms - rainfall) are very highly correlated historically with "terror" attacks and invasions or raids. The data for this can be more easily obtained. But the area of analysis needs to include the "greater India" rim zone too - where the "terror" is sourced from. Rainfall data, and aridity data is easily obtained.

Thanks Bji for the insight. Will add it to the data mapping.

[brihaspati]

Friends are you using the data sets from http://www.start.umd.edu/gtd/search/? It is a good source although with some criticisms. Let me know of your assessments as I use some of it formally.

[BajKhedawal]

RamaY thanks email sent

Brihaspati thanks for the link, i still havent formed a hypothesis yet but most likely i will do something on lines of: GDP of regional countries and their support to terrorism thereof.

[ramana]

A BBC article on high-tech tools being used to monitor traffic from Chile to coordinate releif.

Can some of you guys learn to sue them to understand events like Kabul attack?

Hi Tech tools using Twitter feed etc

[RamaY]

A BBC article on high-tech tools being used to monitor traffic from Chile to coordinate releif.

Can some of you guys learn to sue them to understand events like Kabul attack?

Hi Tech tools using Twitter feed etc

Ramanaji - The Ushahidi tool depends upon people sending SMS or email (or something like that). We may not be able to use it for Kabul attack type scenarios, unless we create a server and provide a number for people to send SMS.

This is my understanding. Appreciate other opinions.

[ramana]

A BBC article on high-tech tools being used to monitor traffic from Chile to coordinate releif.

Can some of you guys learn to sue them to understand events like Kabul attack?

Hi Tech tools using Twitter feed etc

Ramanaji - The Ushahidi tool depends upon people sending SMS or email (or something like that). We may not be able to use it for Kabul attack type scenarios, unless we create a server and provide a number for people to send SMS.

This is my understanding. Appreciate other opinions.

True. What about Swiftriver? and SMS Turk. We need to build BRF volunteers to jump in and add value. We should think more pro-actively and be ready for next incident and not just fill pages and pages of rants like after 26/11.

Or Flighty?

http://swift.ushahidi.com/extend/flighty/

[pgbhat]

Website on Political Economy of Terrorism

Ethan Beuno De Mesquita

An analytical history of terrorism, 1945–2000
by William F. Shughart II
Federation of American Scientists

I took my Graduate level Industrial Organization course under Prof. Shughart. Cool dude. Excellent Professor.

[ramana]

I was reading a chapter in Malcolm Gladwell's latest book "What the Dog Saw" about how by monitoring and analysing Nazi propoganda to their local population the Allies were able to deduce the extent and even the status of the V weapons. The technique has data collection and unbiased analysis along with some psychology. Some of it is documented in a book "Propoganda Analysis" published in the fifties.

Will get the exact title later.

We have wrt TSP data gathering and psychology but are sorely lacking in analysis. Need to beef up that area.

Gladwell Link

The analysts listened to the same speeches that anyone with a shortwave radio could listen to. They simply sat at their desks with headphones on, working their way through hours and hours of Nazi broadcasts. Then they tried to figure out how what the Nazis said publicly—about, for instance, the possibility of a renewed offensive against Russia—revealed what they felt about, say, invading Russia. One journalist at the time described the propaganda analysts as "the greatest collection of individualists, international rolling stones, and slightly batty geniuses ever gathered together in one organization." And they had very definite thoughts about the Nazis' secret weapon.

The German leadership, first of all, was boasting about the secret weapon in domestic broadcasts. That was important. Propaganda was supposed to boost morale. If the Nazi leadership said things that turned out to be misleading, its credibility would fall. When German U-boats started running into increasingly effective Allied resistance in the spring of 1943, for example, Joseph Goebbels, the Nazi minister of propaganda, tacitly acknowledged the bad news, switching his emphasis from trumpeting recent victories to predicting long-term success, and blaming the weather for hampering U-boat operations. Up to that point, Goebbels had never lied to his own people about that sort of news. So if he said that Germany had a devastating secret weapon it meant, in all likelihood, that Germany had a devastating secret weapon.

Starting from that premise, the analysts then mined the Nazis' public pronouncements for more insights. It was, they concluded, "beyond reasonable doubt" that as of November, 1943, the weapon existed, that it was of an entirely new type, that it could not be easily countered, that it would produce striking results, and that it would shock the civilian population upon whom it would be used. It was, furthermore, "highly probable" that the Germans were past the experimental stage as of May of 1943, and that something had happened in August of that year that significantly delayed deployment. The analysts based this inference, in part, on the fact that, in August, the Nazis abruptly stopped mentioning their secret weapon for ten days, and that when they started again their threats took on a new, less certain, tone. Finally, it could be tentatively estimated that the weapon would be ready between the middle of January and the middle of April, with a month's margin of error on either side. That inference, in part, came from Nazi propaganda in late 1943, which suddenly became more serious and specific in tone, and it seemed unlikely that Goebbels would raise hopes in this way if he couldn't deliver within a few months. The secret weapon was the Nazis' fabled V-1 rocket, and virtually every one of the propaganda analysts' predictions turned out to be true.

The political scientist Alexander George described the sequence of V-1 rocket inferences in his 1959 book "Propaganda Analysis," and the striking thing about his account is how contemporary it seems. The spies were fighting a nineteenth-century war. The analysts belonged to our age, and the lesson of their triumph is that the complex, uncertain issues that the modern world throws at us require the mystery paradigm.

[ramana]

Google Book on Content Analysis Reader

[ramana]

Maybe this should go in psy-ops thread also:

Among the more common fallacious techniques or inadequate proofs:
- Raising the volume, increasing the stridency, or stressing the emotionalism of an argument does not improve its validity. This is called argument by exhortation.
- Sequence does not imply causation. If Joan is elected to the board of directors of a bank on May 1, and Raul gets a loan on July 26, further evidence is needed to prove a direct or causal connection.
- Association does not imply agreement, hence the term “guilt by association” has a pejorative meaning. Association proves association: it suggests further questions are appropriate, and demonstrates the parameters of networks, coalitions, and personal moral distinctions - nothing more.
- Participation in an activity, or presence at an event, does not imply control.
- Congruence in one or more elements does not establish congruence in all elements. Gloria Steinem and Jeane Kirkpatrick are both intelligent, assertive women accomplished in political rhetoric. To assume they therefore also agree politically would be ludicrous. If milk is white and powdered chalk is white, would you drink a glass of powdered chalk?
- Similarity in activity does not imply joint activity and joint activity does not imply congruent motivation. When a person serves in an official advisory role or acts in a position of responsibility within a group, however, the burden of proof shifts to favor a presumption that such a person is not a mere member or associate, but probably embraces a considerable portion of the sentiments expressed by the group. Still, even members of boards of directors will distance themselves from a particular stance adopted by groups they oversee, and therefore it is not legitimate to assume automatically that they personally hold a view
expressed by the group or other board members. It is legitimate, however, to assert that they need to distance themselves publicly from a particular organizational position if they wish to dissociate themselves from it.
- Anecdotes alone are not conclusive evidence. Anecdotes are used to illustrate a thesis, not to prove it.
- Lack of evidence is not proof, nor even a suggestion, of a possible conspiracy. Just because an incident lacks an apparently obvious explanation, or a person fails to do something that seems obviously required or effective, it doesn't imply a sinister motive or plot

The Lees developed a list of seven hallmark tricks of the manipulative propagandist:
- Name Calling: hanging a bad label on an idea
- Card Stacking: selective use of facts or outright Falsehoods
- Band Wagon: a claim that everyone like us thinks this way
- Testimonial: the association of a respected or hated person with an idea
- Plain Folks: a technique whereby the idea and its proponents are linked to “people like you and me”
- Transfer: an assertion of a connection between something valued or hated and the idea or commodity being discussed
- Glittering Generality: an association of something with a “virtue word” to gain approval without examining the evidence

[naren]

How about buzz generated in the online community to predict election outcomes ?

- We can assign a score for every major decision taken during the current ruling party's term
- We can quantifiably measure the "buzz" for each event
---- # of blog entries, social network & news site comments containing particular keyword at the time of an event
- Using the above two, we can judge the mood of the herd. Assign a score & predict whether the people favour the current govt or not.

Just a thought.

[naren]

Maybe this should go in psy-ops thread also:

The Lees developed a list of seven hallmark tricks of the manipulative propagandist:
- Name Calling: hanging a bad label on an idea
- Card Stacking: selective use of facts or outright Falsehoods
- Band Wagon: a claim that everyone like us thinks this way
- Testimonial: the association of a respected or hated person with an idea
- Plain Folks: a technique whereby the idea and its proponents are linked to “people like you and me”
- Transfer: an assertion of a connection between something valued or hated and the idea or commodity being discussed
- Glittering Generality: an association of something with a “virtue word” to gain approval without examining the evidence

Sounds exactly like Fox News strategy

[naren]

A BBC article on high-tech tools being used to monitor traffic from Chile to coordinate releif.

Can some of you guys learn to sue them to understand events like Kabul attack?

Hi Tech tools using Twitter feed etc

Ramanaji - The Ushahidi tool depends upon people sending SMS or email (or something like that). We may not be able to use it for Kabul attack type scenarios, unless we create a server and provide a number for people to send SMS.

This is my understanding. Appreciate other opinions.

If I'm not mistaken, every attack is preceded by extensive co-ordination through cell phone. So right before the attack, there must be higher volume of calls at the given location.

Say we keep monitoring the cell phone traffic around a particular location (embassy) for a month. The patterns should be consistent and somewhat predictable over time. So unusually lengthy or high volume of short calls or extensive texting should raise a flag. Though it sounds needle-in-a-stack, I think it should be spottable.

[ramana]

Who is we kemo sabay?

[naren]

^^^

"We" = security team responsible for preventing terrorist attacks at a given location. In our case, it would be the security staff at the Indian embassy in Kabul. They will monitor the cell phone call patterns around the embassy (with the cooperation of local cell companies or may be snoop if not cooperative). They will raise the threat level if they find any anomalies in the aggregate data pattern.

[ramana]

A tool for plotting social networks:

http://www.andrew.cmu.edu/user/krack/krackplot.shtml

[abhischekcc]

The Lees developed a list of seven hallmark tricks of the manipulative propagandist:
- Name Calling: hanging a bad label on an idea
- Card Stacking: selective use of facts or outright Falsehoods
- Band Wagon: a claim that everyone like us thinks this way
- Testimonial: the association of a respected or hated person with an idea
- Plain Folks: a technique whereby the idea and its proponents are linked to “people like you and me”
- Transfer: an assertion of a connection between something valued or hated and the idea or commodity being discussed
- Glittering Generality: an association of something with a “virtue word” to gain approval without examining the evidence

Sounds exactly like Fox News strategy

Sounds exactly like the strategy used by marxist historians in India to attack Hinduism.

[ramana]

Most propaganda techniques look alike. I posted the checklist to make people aware.

[svinayak]

Sounds exactly like the strategy used by marxist historians in India to attack Hinduism.

I wanted to post it before you did. These are well caliberated and Indian elite has really got into it. The western educated Indian sociologists have been trained in these methods and they using their control over the media to change the social behavior in India.
Do a google and you see a lot of material was originated from the Nazi experiments on social control.

http://en.wikipedia.org/wiki/Cult_checklist

Steven Hassan

In Releasing the Bonds: Empowering People to Think for Themselves,[2][3] cult counselor Steven Hassan describes his "BITE model" stating that it is not necessary for every item to be present:

[edit]Behavior Control
Regulation of individual's physical reality
"Where, how, and with whom the member lives and associates, what clothes, colors, hairstyles the person wears, what food the person eats, drinks, adopts, and rejects, how much sleep the person is able to have, financial dependence, little or no time spent on leisure, entertainment, vacations."
Major time commitment required for indoctrination sessions and group rituals
Need to ask permission for major decisions
Need to report thoughts, feelings, and activities to superiors
Rewards and punishments (behavior modification techniques -- positive and negative)
Individualism discouraged; "Group think" prevails
Rigid rules and regulations
Need for obedience and dependency

[edit]Information Control
Use of deception
Deliberately holding back information, distorting information to make it more "acceptable," "outright lying."
Access to non-cult sources of information minimized or discouraged
Media (books, articles, newspapers, magazines, TV, radio), critical information, former members, keep members so busy they don't have time to think and check things out.
Compartmentalization of information; Outsider vs. Insider doctrines
"Information is not freely accessible, information varies at different levels and missions within pyramid, leadership decides who "needs to know" what and when."
Spying on other members is encouraged
"Pairing up with "buddy" system to monitor and control, reporting deviant thoughts, feelings, and actions to leadership, individual behavior monitored by whole group."
Extensive use of cult generated information and propaganda
"Media (newsletters, magazines, journals, audio tapes, videotapes, etc), misquotations, statements taken out of context from non-cult sources."
Unethical use of confession
"Information about "sins" used to abolish identity boundaries, past "sins" used to manipulate and control (no forgiveness or absolution)."


[edit]Thought Control
Need to internalize the group's doctrine as "Truth"
"Adopting the group's map of reality as "Reality" (Map = Reality), Black and White thinking, Good vs. Evil, Us vs. Them (inside vs. outside)."
Use of "loaded" language (for example, "thought-terminating clichés". ). Words are the tools we use to think with.[clarification needed] These "special" words constrict rather than expand understanding, and can even stop thoughts altogether. They function to reduce complexities of experience into trite, platitudinous "buzz words" (The best analogy would be Newspeak in Nineteen Eighty Four by George Orwell).
Only "good" and "proper" thoughts are encouraged.
Use of hypnotic techniques to induce altered mental states
Manipulation of memories and implantation of false memories
Use of thought-stopping techniques, which shut down "reality testing" by stopping "negative" thoughts and allowing only "good" thoughts
Rejection of rational analysis, critical thinking, constructive criticism. No critical questions about leader, doctrine, or policy seen as legitimate
No alternative belief systems viewed as legitimate, good, or useful


[edit]Emotional Control
Manipulate and narrow the range of a person's feelings.
Make the person feel that any problems are always their fault, never the leader's or the group's.
Excessive use of guilt: identity guilt (who you are, not living up to your potential, your family, your past, your affiliations, your thoughts, feelings, actions), social guilt, historical guilt.
Excessive use of fear: fear of thinking independently, fear of the "outside" world, fear of enemies, fear of losing one's "salvation", fear of leaving the group or being shunned by group, fear of disapproval.
Extremes of emotional highs and lows.
Ritual and often public confession of "sins".
Phobia indoctrination: inculcating irrational fears about ever leaving the group or even questioning the leader's authority. The person under mind control cannot visualize a positive, fulfilled future without being in the group.

[svinayak]

Most propaganda techniques look alike. I posted the checklist to make people aware.

There is something special you need to be familiar and is used in India with English language media

http://en.wikipedia.org/wiki/Newspeak

Newspeak is a fictional language in George Orwell's novel Nineteen Eighty-Four. The term was also used to discuss Soviet phraseology.[1] In the novel by Orwell, it is described as being "the only language in the world whose vocabulary gets smaller every year". Orwell included an essay about it in the form of an appendix[2] in which the basic principles of the language are explained. Newspeak is closely based on English but has a greatly reduced and simplified vocabulary and grammar. This suits the totalitarian regime of the Party, whose aim is to make any alternative thinking—"thoughtcrime", or "crimethink" in the newest edition of Newspeak—impossible by removing any words or possible constructs which describe the ideas of freedom, rebellion and so on. One character, Syme, says admiringly of the shrinking volume of the new dictionary: "It's a beautiful thing, the destruction of words."

The basic idea behind Newspeak is to remove all shades of meaning from language, leaving simple dichotomies (pleasure and pain, happiness and sadness, goodthink and crimethink) which reinforce the total dominance of the State. Similarly, Newspeak root words served as both nouns and verbs, which allowed further reduction in the total number of words; for example, "think" served as both noun and verb, so the word thought was not required and could be abolished. A staccato rhythm of short syllables was also a goal, further reducing the need for deep thinking about language. (See duckspeak.) Successful Newspeak meant that there would be fewer and fewer words – dictionaries would get thinner and thinner.

In India party has put a thought crime on Hindu nationalism
http://en.wikipedia.org/wiki/Thoughtcrime

In the dystopian novel Nineteen Eighty-Four by George Orwell, a thoughtcrime is an illegal type of thought.
In the book, the government attempts to control not only the speech and actions, but also the thoughts of its subjects, labelling disapproved thought as thoughtcrime or, in Newspeak, "crimethink".[1]

Technology played a significant part in the detection of thoughtcrime in Nineteen Eighty-Four—with the ubiquitous telescreens which could inform the government, misinform and monitor the population.

[RamaY]

I am reading the "predectioneer's game" and the other presents the game for Pakistan on two recent developments; Musharraf's decline and amount of USAID required to make pakis fight N.Vaziristan battle against AQ, Pakibans, and Talibans.

My conspiracy mind somehow sees the path from Musharraf's decline to Current military/political goes thru BB's headroof liver.

Gurulog - is there any workable connection between zardari and sharrifs? What would be BB's stand on fighting US's fight, if she were to miss that roof-lever? What would be her price in terms of aid and strategic concessions? Could she work hand hand with nawaz and Kiyanahi?

Appreciat some insight!

[ramana]

Updates from Prof. Mesquita's Blog

How are things going in Pakistan? The analysis in the penultimate chapter of The Predictioneer’s Game indicated that IF — a contingent forecast — the US gave Pakistan $1.5 billion in aid then the Pakistani government would turn away from making side deals with the Taliban and Al Qaeda militants in Pakistan and, instead, go after them but not wholeheartedly. This is what has been happening. The Pakistani regime is now doing much more to rein in the militants but it has explicitly refused to go after the Pakistani Taliban leader (who you can see sitting next to the Jordanian doctor who murdered 7 US CIA agents in Afghanistan as well as one Jordanian agent). Going after him apparently is too politically costly for the Pakistani leaders. It will be interesting to see whether this changes given his video appearance. Anyone care to try to model that? Please do and post your results.

Before the aid deal was approved, the Pakistani government had negotiated a partial deal with the Taliban who had taken over Swat. Now they are fighting them. Looks like my students back in 2008 pretty much got it right.

[ramana]

An insightful paper on modeling threats to homeland

Building a Contingency Menu

[ramana]

A few pdfs on Social Network theory with applications:


Sandia Paper:

Computational Social Dynamic Modeling of Group Recruitment

Science paper on

Network Theory and emergence of the creative enterprise
Read the conclusions about how a gathering of like minded people will end up in group think!

and

Can we personally influence the future with our present resources?

[RamaY]

KS garu in one of his columns wrote that while browsing in a London used book sore he picked up a American book on crisis game theory which had the exact Pak moves in 1965 gamed with participation of Pak officers in a US military academy and was appalled at the perfidy.

Could be the 1965 war was supposed to trigger some other events and India as usual stymied them by not responding per the game scenario.
1965 war was accompanied by Indonesia claiming the Indian Ocean name.
KSA and Gulf changing their money from Indian rupee to Dollar.
Two years later OIC was created...

Wow

Ramanaji, I will also actively google for that. But if you can pull that article out of your memory, please provide the book details.

{Added Later}
Could it be this?
The American military and political establishment had concluded that in case of a war, Pakistan would win.
The Pentagon and Harvard University played a war game at the Institute of Defence Analysis, Washington, DC, in March 1965. The war game and its results were available in a book, Crisis Game by Sidney Giffin, by the spring of 1965.

[ramana]

Yes "Crisis Game" Sidney Griffen is the book he referred to.

[ramana]

From Nightwatch 10 August 2010

Note to new analysts: Analysts and their supervisors need to remember that every judgment must withstand the three tests of the scientific method:

Auditability -- the evidentiary path must be transparent and traceable by independent, unbiased analysts.

Replicability - the findings can be capable of being replicated from the evidence by independent, unbiased analysts, even if they disagree with the findings.

Irrefutability - no evidence must exist that absolutely refutes and negates the findings, such as physical impossibility or independent admissions of causality that were not addressed.


If the Allied findings meet the criteria of the scientific method, there is nothing to fear from a biased North Korean set of findings and a lot to be gained.

[ramana]

Afterr the 2008 global financial meltdown, a lot of books on figuring out the truth in data are coming out. The books are diverse fields like finance, political analysis and intelligence analysis.

[ramana]

A while back maybe after the Lok Sabha attack, B Ramanji started a thread now long gone about signals and how to assess them.

I have been on look out for books on that since its not an India unique problem.

So I found this book:

Clayton Thyne: How International Relations Effect Civil Conflict, Cheap Signals and Costly consequences.

[ramana]

Nightwatch gives pointers on how to interpret Korean COnfontation from pubilc data. Off course its after the event but is more like backwards projection.

Nightwatch 2 Dec 2010

The Korean Confrontation
South Korea: South Korean media have spotlighted a significant failure by South Korean intelligence agencies. The state intelligence agency and the military came under fire Thursday after it became known that they did not take appropriate precautionary measures In August after they detected signs of a possible North Korean attack on the five offshore islands in the Yellow Sea.

National Intelligence Service director Won Sei-hoon told a parliamentary intelligence committee Wednesday that the agency confirmed the possibility of a North Korean attack on the islands through wiretapping, according to lawmakers who attended the closed-door session. Won, however, was quoted by the lawmakers as saying, "(The agency) did not expect the North to launch an attack on civilians as it has routinely shown menacing words and behaviors. The military authorities judged that the North could mount an attack just south of the maritime border."

Comment: This is a significant intelligence analytical failure for South Korea and the US. They share intelligence warning methods but failed to use them to any good effect. Director Won's testimony confirms the judgment by Professor Bruce Bechtol last week that the 23 November attack was long planned, deliberate and predatory. Moreover, South Korean and US intelligence agencies had more than enough intelligence evidence to justify issuing an intelligence warning as early as early August.

NightWatch attests to the past proficiency of South Korean, USFK and US JCS, J2, intelligence analysts in using structured techniques, driven by intelligence evidence, to reach accurate predictions about North Korean behavior more than 90% of the time through 2005. This time no one warned.

If the South Koreans persuaded themselves there was no threat, the whole US intelligence system would defer to them. This is precisely what has happened in dealing with Israel - for example, the 1973 Yom Kippur War and the 2006 War in Lebanon. When the intelligence services of anAllied country most at risk got it wrong, so did US intelligence services, with a few singular exceptions. That apparently is what happened this time too, but it is not an inevitable result and defeats the purpose of allied sharing with independent analysis.

It is a mark of incompetence in intelligence analytical judgment and management that inappropriate past examples were used to minimize the real time, salient intelligence indicators of attack. There was more than enough evidence on which to base a warning judgment and have it disseminated on both sides of the Pacific Ocean. That is the key take away from Director Won's testimony. See Neustadt and May, Thinking in Time, for education in the "appropriate" uses of history in national security decision making.

The evidence, according to The Korea Herald and other news outlets, included firing exercises - aka rehearsals -- unit movements, introduction of new and different units and intercepts of North Korean orders for the gun units to be ready to fire.

In addition, had an analyst taken the time to search for anomalies in North Korean media, such as an August commentary about not respecting the legal status of the islands and references to physical retaliatory strikes, he or she should have been able to match intent with capabilities and opportunity, as discussed in past issues of NW, to establish at least a 75% probability of an attack as early as early August with the timing to be determined.

An increase in military readiness on the five islands from August onward would have been low cost and contributed to minimal damage, quick response and possibly even deterrence, in contrast to what happened on 23 November 2010. There was enough time that South Korea could have trapped the North.

Analysts apparently no longer know how to distinguish North Korean specific intentions based on behavior. That was once a strong point of the Korean NIS, the US National Warning Staff and the Directorate of Intelligence, J2, Joint Staff in the Pentagon. The skills, training and use of proven diagnostic and predictive warning techniques evidently have lapsed and, as a result, people died for lack of warning. Sixty years after the Korean War, analysts seem to have forgotten warning of attack. Warning remains the prime directive for all US intelligence since the National Security Act of 1947.

Feedback: A brilliant and perceptive Reader noted that the deployment of air defense missile units to Yeonpyeong Island, which puts them within range of North Korean artillery and rocket fire, is primarily a political and public relations gesture that has slight military purpose.

North Korea: The Korea Central News Agency reported that leader Kim Jong-il made a guidance visit to three plants and facilities in Tanjon City on 2 December. His sister, brother-in-law and other top officials accompanied him but not his heir apparent.

Comment: This is Kim's second reported public appearance since the end of the US-South Korean naval exercises. On 1 December, KCNA reported him as visiting Hamhung University, on the east coast. Kim has several lavish villas near Hamhung.

The two reports of public appearances, albeit undated, reinforce the image that North Korea remains in a condition of normality and the leadership does not intend to escalate the confrontation over the North's shelling of Yeonpyeong Island.

Shades of Kargil and even 26/11!

[ramana]

X-post...
Has anyone done an organizational analysis of the TSP terrorist groups? What is their org structure and where are the nodes? Its clear they have cells in India too but how are they organised? Also on a generic note what type of organisation does the ISI prefer for its irregular assets?

Can someone do an analysis of the KPS Gill portal data and see what emerges?

[Pratyush]

PM's New Year resolution: Concentrate on the neighbourhood

NEW DELHI: After entertaining almost 25 heads of state and government in 2010, the Manmohan Singh government's New Year resolution is to concentrate attention on India's immediate neighbours, Indian Ocean and Africa in 2011. Top billing will be given to Nepal and Bangladesh, interestingly, not Pakistan.

Nepal is in a politically catatonic state, with neither the political parties nor the Maoists willing to work on a compromise, leaving the country with virtually no government. With a new ambassador, Jayant Prasad, ready to drive Indian policy in Kathmandu, India will try to push for "free and fair" elections in the Himalayan nation -- where the Maoists contest elections after disarming, so that nobody has an unfair advantage. That has to be the crux of the political deal in Kathmandu, but it has to be done by the Nepalis themselves.

Sources said, India has been approached by different shades of opinion in Nepal asking for intervention, but India continues to hold aloof, because active intervention doesn't make friends among the Nepalis. The foreign minister, SM Krishna is scheduled to visit Kathmandu soon __ after the Maoist leader Baburam Bhattarai visits India next week. But the Indian message remains the same. Maoists have to disarm, following which there can be a political understanding. Bhattarai's visit will be important, for first-hand assessment of current Maoist thinking.

With Bangladesh, the news is a lot more positive. 2010 was a good year for the India-Bangladesh brief, largely due to the proactive role played by the Bangladeshis. In many respects, India has to step up to the plate in the coming year, follow through on the commitments made to Dhaka. Bangladesh wants greater economic openness by India, as well as an equitable agreement on border demarcation. All of this is doable, and at the highest level, the government has put the word out that Bangladesh gets priority. India is hoping to put high level visits by Krishna and perhaps even PM Manmohan Singh.

Pakistan prompts only dismal head shakes in the government. PM has no real maneuvering space on Pakistan, and Pakistan is not ready to play ball either. Yet India continues to get battered by the rest of the world to resume dialogue with Pakistan. Pakistan foreign minister Shah Mehmood Qureshi is scheduled to visit India, and is likely to do so in the next couple of months. But nobody makes any promises on any substantive movement with Pakistan. Of course, another terror attack of moderate intensity will sweep all bets off the table.

Manmohan Singh plans to correct his dismal record of engagement with Africa in 2011. An India-Africa summit is on the cards, though the venue in Africa is yet to be worked out, said sources. Moreover, South Africa is expected to host the IBSA (India, Brazil South Africa) summit in the coming year, and India should be able to plan a number of visits around this event. India has hosted numerous African leaders over the past couple of years, so "now we owe everybody a visit" observed a senior official.

After the Barack Obama visit, India and the US have pledged to work together in Africa. The contours of this engagement will revolve around a couple of things __ replicating India's green revolution experience in Africa and leap-frogging Africa's industrialization to make their exports more relevant in the 21st century. As a number of African leaders have told Singh, they didn't want their countries to be merely sources of raw materials for industrialized nations (read China). So development of capacities, capabilities etc will all be part of India's Africa outreach.

[ramana]

A good book to read:

http://en.wikipedia.org/wiki/A_Whole_New_Mind

[ramana]

Couple of good reviews with common theme

The Next Nostradamus (2008)

The Next Nostradamus will explore the long and enigmatic history of predicting the future as well as the science and groundbreaking technology being used today to try and uncover what tomorrow will bring. Five thousand years ago, the Mayans studied astrology - and predicted the end of the world in 2012. In the mid-sixteenth century, Nostradamus used a crystal ball to make a haunting reference to the September 11 attacks on the Twin Towers in his predictions of a possible World War III. And today scientists such as New York University's Bruce Bueno de Mesquita utilize cutting edge science to predict the outcome of virtually any international conflict with alarming accuracy. From predicting the second Intifada two years before it happened in 2000 to warning of a violent crackdown on dissidents by the Chinese government just four months before Tiananmen Square in 1989, Bueno de Mostquita himself uses a complex algorithm to foresee the future...with 97% accuracy. And his predictions for what comes next are destined to join those of the Mayans and Nostradamus before him.

and

Bruce Bueno de Mesquita, "The Logic of Political Survival"
Publisher: The MIT Press | 2003 | ISBN 0262025469 | 551 pages |

The authors of this ambitious book address a fundamental political question
: why are leaders who produce peace and prosperity turned out of office while those who preside over corruption, war, and misery endure? Considering this political puzzle, they also answer the related economic question of why some countries experience successful economic development and others do not.
The authors construct a provocative theory on the selection of leaders and present specific formal models from which their central claims can be deduced. They show how political leaders allocate resources and how institutions for selecting leaders create incentives for leaders to pursue good and bad public policy. They also extend the model to explain the consequences of war on political survival. Throughout the book, they provide illustrations from history, ranging from ancient Sparta to Vichy France, and test the model against statistics
gathered from cross-national data. The authors explain the political intuition underlying their theory in nontechnical language, reserving formal proofs for chapter appendixes. They conclude by presenting policy prescriptions based on what has been demonstrated theoretically and empirically.

[ramana]

Building on idea in the Managing TSP failure thread

Links on Options market:

http://www.intrade.com/

Wiki description:
http://en.wikipedia.org/wiki/Intrade

....
Intrade is a prediction market which allow individuals to take positions (trade 'contracts') on whether future events will or will not occur. An example event is a political election, which is almost always settled in a well-defined and easily verifiable manner. The contract might be "Barack Obama to win 2012 U.S. presidential election." Other events include financial predictions, such as "NASDAQ Average to close higher today." Intrade facilitates trades between members, charging varying commissions and fees on trades, but does not participate in trading itself.

Trading positions are provided in the common nomenclature of long (will happen) and short (will not happen). The trading unit is a contract with a notional settlement value, typically $10, and the contract may trade in range of 0-100 where 1 point equals US$0.10 in value. If the event specified in a given contract occurs, the contract settles at 100 points or $10; otherwise, the contract settles at 0 or $0 in value. Thus, the current price of the contract can be imputed as the market's global opinion of the probability that the specified event will occur.

Because most events take place over a well-defined time span, traders can trade both before and during an event. The exchange likes to have dedicated market makers in its benchmark markets to provide a two way price at almost all times.

Intrade provides both real money (Intrade.com) and play money (Intrade.net) prediction market trading.[3] The minimum deposit for opening a real money account is US$25.00, but US$100 is recommended to "fully appreciate the trading experience".[4]

...

[ramana]

The Egypt crisis has brought out a whole set of tools that failed. I think its the people who used them that failed to see the gaps.


Wired magazine reports:

Pentagon software fails to predict Egypt

It has mention of Bruce Beuno De Mesquita's methods and others.

Pentagon’s Prediction Software Didn’t Spot Egypt Unrest
By Noah Shachtman February 11, 2011 | 7:00 am | Categories: DarpaWatch


In the last three years, America’s military and intelligence agencies have spent more than $125 million on computer models that are supposed to forecast political unrest. It’s the latest episode in Washington’s four-decade dalliance with future-spotting programs. But if any of these algorithms saw the upheaval in Egypt coming, the spooks and the generals are keeping the predictions very quiet.

Instead, the head of the CIA is getting hauled in front of Congress, making calls about Egypt’s future based on what he read in the press, and getting proven wrong hours later. Meanwhile, an array of Pentagon-backed social scientists, software engineers and computer modelers are working to assemble forecasting tools that are able to reliably pick up on geopolitical trends worldwide. It remains a distant goal.

“All of our models are bad, some are less bad than others,” says Mark Abdollahian, a political scientist and executive at Sentia Group, which has built dozens of predictive models for government agencies.

“We do better than human estimates, but not by much,” Abdollahian adds. “But think of this like Las Vegas. In blackjack, if you can do four percent better than the average, you’re making real money.”


Over the past three years, the Office of the Secretary of Defense has handed out $90 million to more than 50 research labs to assemble some basic tools, theories and processes than might one day produce a more predictable prediction system. None are expected to result in the digital equivalent of crystal balls any time soon.

In the near term, Pentagon insiders say, the most promising forecasting effort comes out of Lockheed Martin’s Advanced Technology Laboratories in Cherry Hill, New Jersey. And even the results from this Darpa-funded Integrated Crisis Early Warning System (ICEWS) have been imperfect, at best. ICEWS modelers were able to forecast four of 16 rebellions, political upheavals and incidents of ethnic violence to the quarter in which they occurred. Nine of the 16 events were predicted within the year, according to a 2010 journal article [.pdf] from Sean O’Brien, ICEWS’ program manager at Darpa.

Darpa spent $38 million on the program, and is now working with Lockheed and the United States Pacific Command to make the model a more permanent component of the military’s planning process. There are no plans, at the moment, to use ICEWS for forecasting in the Middle East.

ICEWS is only the latest in a long, long series of prediction programs to come out of the Pentagon’s way-out research shop. Back in the early 1980s, products from a Darpa crisis-warning system program allegedly filled President Reagan’s daily intelligence briefing, with uncertain results. In the late ’80s, analyst Bruce Bueno de Mesquita began his modeling work. According to The New York Times Magazine, Bueno de Mesquita picked Iranian leader Ayatollah Khomeini’s successor five years ahead of time, and forecast Pakistani President Pervez Musharraf’s ouster — to the month.

One former CIA analyst claims that Bueno de Mesquita was “accurate 90 percent of the time.” It’s an assertion that no one — inside the government or out — has independently verified. Perhaps someone at the CIA really is relying on the model, and it really is that good. That hasn’t stopped the agency from swinging and missing for decades on Middle East intelligence estimates.

In 2002, the military’s National Defense University began tapping Abdollahian and his “Senturion predictive political simulation model” to forecast unfolding events in Iraq. According to Abdollahian, the model accurately predicted that Bush administration favorite Ahmed Chalabi would prove to be a lousy ally, and that both Sunni and Shi’ite insurgencies would grow to seriously challenge U.S. forces.

Both Abdollahian and Bueno de Mesquita take a similar approach to the prediction game. They interview lots and lots of experts about the key players in a given field. Then they program software agents to replicate the behavior of those players. Finally, they let the agents loose, to see what they’ll do next. The method is useful, but limited. For every new situation, the modelers have to interview new experts, and program new agents.

A second approach is to look at the big social, economic and demographic forces at work in a region — the average age, the degree of political freedom, the gross domestic product per capita — and predict accordingly. This “macro-structural” approach can be helpful in figuring out long-term trends, and forecasting general levels of instability; O’Brien relied on it heavily, when he worked for the Army. For spotting specific events, however, it’s not enough.

The third method is to read the news. Or rather, to have algorithms read it. There are plenty of programs now in place that can parse media reports, tease out who is doing what to whom, and then put it all into a database. Grab enough of this so-called “event data” about the past and present, the modelers say, and you can make calls about the future. Essentially, that’s the promise of Recorded Future, the web-scouring startup backed by the investment arms of Google and the CIA.

But, of course, news reports are notoriously spotty, especially from a conflict zone. It’s one of the reasons why physicist Sean Gourley’s much heralded, tidy-looking equation to explain the chaos of war failed to impress in military circles. Relying on media accounts, it was unable to forecast the outcome of the 2007 military surge in Iraq.

ICEWS is an attempt to combine all three approaches, and ground predictions in social science theory, not just best guesses. In a preliminary test, the program was fed event data about Pacific nations from 2004 and 2005. Then the software was asked to predict when and where insurrections, international crises and domestic unrest would occur. Correctly calling nine of 16 events within the year they happened was considered hot stuff in the modeling world.

But it doesn’t even meet the threshold that O’Brien, the Darpa program manager and long-time military social scientist, set for strong models. If “we cannot correctly predict over 90% of the cases with which our model is concerned,” he writes, “then we have little basis to assert our understanding of a phenomenon, never mind our ability to explain it.”

Note it has some other pointers which are to be read to understand the pitfalls.

Fool Proof formula to model insurgency

and

Senturion:Predecitive model for Political simulation

And a tool that combines most of these methods:

Crisis Early Warning Prediction tool


cheers!