Hackede CRU-mails

kfl skrev:
Hockey stick..

....

På et tidspunkt - når jeg har mere tid - skal jeg nok redegør for de metoder man kan anvende til rekonstruktion af temperaturkurver, nemlig partial least square regession og principal component regression. Desuend skal jeg have fundet R-koden og data på nettet. Det kan godt tage noget tid.

Mike

At 09:57 22/12/98
>Hi again Mike.
I read
>your Science paper (it was science wasnt it?) about the need to adapt with
>great interest.
...
>If I get your figures right, we would need about 150,000 pounds just to
>produce the leaflets for 50 countries? We dont have that much available,
>but I think I might possibly be able to come up with the money for the
>analysis, and would love to do this by COP5. What do you think?
>
>Cheers, A
***********************************************
>> From: Mike Hulme <xxx@uea.ac.uk>
>> To: markham/dompka <xxx@erols.com>
>> Subject: Re: Great work
>> Date: Friday, December 18, 1998 11:03 AM
>>
>> Adam,
>>

...
Our UKCIP98 report has had a huge impact
>in
>> the UK in mobilising opinion (in fact even as I write, I've had a call
>from
>> the Dept. of Health which, having seen our scenarios, now want a
>> brainstorming workshop to advise the Minister!) and it could have
similar
>> impacts in other countries. The scenarios would all be IPCC-compatible
>so
>> link in with IPCC statements.
>>
>> Of course the reason for raising this is too see where we might get
funds
>> from.

from: philip.brohan@metoffice.gov.uk
subject: Re: Challenge
to: Caspar Ammann <ammann@ucar.edu>

Caspar et al.

A few comments on the proposal.

1) Reconstructions of the real climate differ widely in their results -
they can't all be correct, and it's likely that all existing
reconstructions are in error in some important aspect. ....

The first blogger to break the Climategate2 story has had a visit from the police and has had his computers seized. Tallbloke's Talkshop first reported on CG2 due to the timing of the release being overnight in the USA...

The quality of the global models are too poor to give any clear information about regional climate change...

Dear Tas, Thanks for the email. Steve McIntyre hasn't contacted me directly about Law Dome (yet), nor about any of the series used in the 1998 Holocene paper or the 2003 GRL one with Mike. I suspect (hope) that he won't. I had some emails with him a few years ago when he wanted to get all the station temperature data we use here in CRU. At that time, I hid behind the fact that some of the data had been received from individuals and not directly from Met Services through the Global Telecommunications Service (GTS) or through GCOS.

"..What I think Courtillot wants is our raw station temperature database. We have
entered into agreements to get this data with many National Met Services
and also scientists around the world (dating back to the 1980s). We can use the data in
the gridded
products (see above) but not make the raw station data available.
You can get much of the raw data from a web site in the US (NCDC Asheville)
if you know what you're doing.
This isn't an issue for almost all climate scientists around the world. They are happy
with the products we put together. We've started getting requests from people
in the last few years asking for the raw station data. We've always not made
the raw data available.

Cheers
Phil
"

Hi Ben and Phil, as you may know (Phil certainly knows), I'm on the editorial board of IJC. Phil is right that it can be rather slow (though faster than certain other climate journals!). Nevertheless, IJC really is the preferred place to publish (though a downside is that Douglass et al. may have the opportunity to have a response considered to accompany any comment).
I just contacted the editor, Glenn McGregor, to see what he can do. He promises to do everything he can to achieve a quick turn-around time (he didn't quantify this) and he will also "ask (the publishers) for priority in terms of getting the paper online asap after the authors have received proofs". He genuinely seems keen to correct the scientific record as quickly as possible.
He also said (and please treat this in confidence, which is why I emailed to you and Phil only) that he may be able to hold back the hardcopy (i.e. the print/paper version) appearance of Douglass et al., possibly so that any accepted Santer et al. comment could appear alongside it. Presumably depends on speed of the review process. If this does persuade you to go with IJC, Glenn suggested that I could help (because he is in Kathmandu at present) with achieving the quick turn-around time by identifying in advance reviewers who are both suitable and available. Obviously one reviewer could be someone who is already familiar with this discussion, because that would enable a fast review - i.e., someone on the email list you've been using - though I don't know which of these people you will be asking to be co-authors and hence which won't be available as possible reviewers. For objectivity the other reviewer would need to be independent, but you could still suggest suitable names.

It is rather odd that the email said the two had rerun his (Mann's) exact analysis and got quite different results. I know I couldn't do this, as when Keith, Tim and me wanted to do some comparisons with MBH98 a few years ago a few of the series could not be made available. I'm not trying to make any sort of point here, just to state that repeating an analysis with exactly the same data is normally very difficult.

Phil Jones wrote:
>
> Dear All,
> There are several issues you should be aware of:
>
> 1. UEA has denied access to the data to McIntyre (and at least two
> others in the past) - in 2007. One of the three appealed and that
> appeal was rejected.
> We would look stupid if you released the data now. I can put your FOI
> person
> in touch with the one at UEA. I think they already know each other!
>
...
> 2. I have signed agreements with some Met Services (European ones) in
> the 1990s
> that I would not pass on their data to third parties. The data could be
> used in the
> gridding though and gridded products made available. I never kept a
> list of which stations
> these were though, as I never thought such problems would arise.
>
> 3. Work on the land station data has been funded by the US Dept of
> Energy, and I
> have their agreement that the data needn't be passed on. I got this in
> 2007.
..
> 5. The data aren't yours to release! Maybe there is no formal IPR
> agreement, but there
> is an implicit one.>
...
>
> Cheers
> Phil

- What about CO2 reconstructions for the Holocene that do not derive from
ice-cores? I am thinking of stomatal density derived CO2 records. They would fit into 1.5
but I am not sure whether the ice-core lobby will be happy to deal with them as they tend
to show somewhat different results.

Jones til Osborn mfl:

> Solar Constant now is 1361 W/m*m! AR5 runs will go with
> whatever they have now, as it will take modelling centres
> 1-2 years to retune their models. Most have 1366-1370!
>
> Cheers
> Phil

date: Tue Oct 24 15:20:04 2000 from: Phil Jones
subject: Re: JOC Review to: Brendaw Morris

Dear Brendaw, My review of the paper JCL 3435 is attached. My recommendation is to accept the paper subject to minor changes. I don't wish to see it again. If there are any problems with the attachment, let me know and I can fax the 2 pages.

Cheers Phil Jones

At 06:58 PM 10/7/00 -0400, you wrote: Professor Michael Mann, Editor of Journal of Climate, has suggested you as a possible reviewer of a paper entitled "Differential ENSO and volcanic effects on surface and tropospheric temperatures" (JCL-3435 by T. M. L. Wigley and B. D. Santer.

Glad to hear from Tom you've been writing up your CCSP chapter and extending
it significantly. He gave me a brief summary. I signed off yesterday on the CCSP
report. You should be getting it through Tom Karl later today, or by Monday. As I did
Ch 5, if you want to check anything with me feel free to. I wasn't able to stop some
comments being put in by Lindzen, but Tom has a paper as does Myles which are
enough to ignore his and the Douglass papers.
Cheers
Phil

Forekommer ret normalt at man sådan stopper kommentarer fra dem man ikke er enige med....

date: Mon Apr 14 17:15:21 2003
from: Keith Briffa
subject: NEED HELP WITH 2 REVIEWS
to: m.kelly@uea.ac.uk

I need you to review a couple of papers for me as soon as possible ( to get me out of a muddle) . I believe I gave you one some time ago ( by Ogurtsov et al ( on solar influence on climate ) which I think will be a rejection but I need hard justification .
The other is a short paper on sea ice around Svalbard. PLEASE come in and get them off me and do then straight away. I will not ask you to double mark a load of Climate Change essays in exchange! Seriously though - this will be a big help . I am trying to dig myself out of a
hole (backlog) with theses things so your help would be much appreciated.

--
Professor Keith Briffa,
Climatic Research Unit
University of East Anglia
Norwich, NR4 7TJ, U.K.

Dear Prof. Jones,

My name is Arthur Edelstein, and I am a graduate student in physics at UC Berkeley. I am writing to request the complete HadCRUT3 dataset and computer source code that you and your coauthors used to adjust and analyze the surface temperature data in the following paper:

Brohan, P., J.J. Kennedy, I. Harris, S.F.B. Tett and P.D. Jones, 2006: Uncertainty estimates in regional and global observed temperature>changes: a new dataset from 1850. J. Geophysical Research 111, D12106 http://www.pnas.org/cgi/content/abstract/103/39/14288

I and others would like to understand all details of your analysis of global temperature trends. I am making this request for the surface records you have used and the computer programs that analyze them, consistent with the American Geophysical Union policies, copied below, which apply to the Journal of Geophysical Research.

Thank you very much in advance.

Sincerely, Arthur D Edelstein

From: Phil Jones
Date: Wednesday, October 10, 2007 12:37 am Subject: Fwd: data and source code request To: Michael Wehner Cc: philip.brohan@metoffice.gov.uk

Hi Mike,

Can you do a bit of discrete looking at UC Berkeley to see if this student really is a student in Physics? I'm planning to ignore the request, but am a little curious as to who the supervisor may be. I don't think I would have had the nerve to send a request like this when I was a student. I don't think I'd have the nerve to send one this blunt even now. It seems a pointless PhD. I would have thought that Berkeley would be above this sort of thing.

No rush if you're weighed under with proper work!

Cheers Phil

Frankly, I would simply put the whole CRU database (in an
as-impenetrable-as-possible form) up on the web site along with a brief
history of it's provenance (and the role of the NMSs) and be done with
it. If specific NMS contracts forbid posting of their raw data, then he
should remove the ones that he contractually can't post and direct
peoples attentions to the NMS's concerned. Why should Phil be the fall
guy for nutty 'commercial' restrictions imposed by various governments?

Bottom line: This isn't going to stop.

Gavin

Just for background in case of any questions, here's the GISS press release below. GISS has 2007 higher in rank than we do, just like 2005 - which they have top, instead of 1998. A quick look at their 2007 spatial map indicates why - their extrapolation of the warm Arctic coastal data across the Arctic Ocean. Despite the differences in base periods, the maps (our Fig 3a and their 1b) are quite similar over most of the world. Their extrapolation also makes Antarctica much warmer than it probably was.

Meanwhile, I suspect you've both seen the latest attack against his Yamal work by McIntyre. Gavin and I (having consulted also w/ Malcolm) are wondering what to make of this, and what sort of response---if any---is necessary and appropriate. So far, we've simply deleted all of the attempts by McIntyre and his minions to draw attention to this at RealClimate.

Man sletter ganske enkelt indlæg. Intet nyt overhovedet, men meget sjovt at se det så casual omtalt

Man sletter ganske enkelt indlæg. Intet nyt overhovedet, men meget sjovt at se det så casual omtalt

...That's the one where he goes on to say

I've been trying to no avail to get some journalist to look into their funding, industry connections, etc. they need to be exposed–badly!

Unfortunately for Mike it was he who was exposed badly

I've read Steig et al now, and I can see all the comments on the CA and RC sites about some of the data. It seems that BAS have made some mistakes with some of the AWS sites. The only AWS site used in CRUTEM3 is the one at Byrd, as this is at one of the manned sites. The issue with the AWS's is getting reasonable data in real time. Whilst I was away the checked monthly data arrived for 2002! I will add Byrd's data in. The problem is that some sites get buried, but still seem to transmit.

"
...
Of course, this doesn't resolve any fundamental methodological concerns
about the S et al. approach that may exist but it does give us a
reasonable chance of creating a much more homogeneous READER manned
station dataset for next IPCC AR and our future products.
"

From: Jonathan Jones...
To: Palmer Dave Mr
Cc: Heath Robert Mr ; Baker Jane Mrs
Subject: RE: Environmental Information Regulations 2004 request (FOI_09-117; EIR_09-14)
- Response
Dear Mr Palmer,
Thank you for your letter dated 14 August, reference ENVIRONMENTAL INFORMATION
REGULATIONS 2004 - INFORMATION REQUEST (FOI_09-117; EIR_09-14) in response to my request
for "a copy of any digital version of the CRUTEM station data set that has been sent
from CRU to Peter Webster and/or any other person at Georgia Tech between January 1,
2007 and June 25, 2009".
I regret that I do not consider your response satisfactory, and am therefore appealing
your decision. As I understand you are currently on holiday I am copying this to Bob
Heath (r.heath@uea.ac.uk) and Jane Baker (jane.baker@uea.ac.uk) as you requested in your
vacation message.
You have refused my request on three grounds, all of which are incorrect.
1. Reg. 12(4)(b) - Request is manifestly unreasonable: Information is available
elsewhere. You claim that "the requested data is a subset of data already available from other
sources" namely the gridded data made available by the GHCN and the CRU. It is
factually incorrect to claim that "the requested data is a subset of data already
available from other sources" and your argument cannot stand.

A "subset of data already
available" would mean that the data I requested could be obtained from "the gridded data
made available by the GHCN and the CRU" by downloading some or all of this data and
deleting selected parts. The data I have requested cannot be obtained in this manner.
I refer you to the discussion of the gridding process at
[3]http://www.cru.uea.ac.uk/cru/data/landstations/ .
You further claim that "it is unreasonable for the University to spend public resources
on providing information in a different format to that which is already available". However I asked for "a copy of any digital version of the CRUTEM station data set that
has been sent from CRU to Peter Webster and/or any other person at Georgia Tech between
January 1, 2007 and Jun 25, 2009". I have only requested a copy of a data set which has
already been prepared by the university, and so is already available. Once again your
statement is factually incorrect and your argument cannot stand.

2. Reg. 12(5)(a) - Adverse effect on international relations: Release would damage
relations with scientists & institutions from other nations
3. Reg. 12(5)(f) - Adverse effect on the person providing information: Information is
covered by a confidentiality agreement
I will take these two points together as they are in essence the same. I begin by
noting that it is wholly perverse to claim simultaneously that the data is "already
available" and that the data is "confidential". Clearly these two statements cannot
simultaneously be true. With regard to Reg. 12(5)(a) you state that releasing this information "would damage the
trust that other national scientists and institutions have in UK-based public sector
organisations" and consequently "would damage the ability of the University and other UK
institutions to co-operate with meteorological organisations and governments of other
countries". I draw your attention to resolution 40 of the World Meteorological
Organization which states that "WMO commits itself to broadening and enhancing the free
and unrestricted international exchange of meteorological and related data and
products". It is perverse to claim that acting in accordance with this resolution could
endanger cooperation with meteorological organizations.


With regard to Reg. 12(5)(f), the data I requested has already been provided to at least
one other individual, namely Peter Webster at Georgia Tech. Clearly this data cannot be
covered by a strict confidentiality agreement.
It is, of course, true that this data could be covered by limited confidentiality
agreements. The FOI and EIR are quite clear on the responsibilities of organizations
claiming exemption on grounds of confidentiality. The exemption "only applies if a
breach of confidence would be 'actionable'".

Courts will only recognise that a person
holds information subject to a duty of confidence in two types of situations:
a) where that person expressly agrees or undertakes to keep information confidential:
there is an express duty of confidence
b) where the nature of the information of the circumstances in which the information is
obtained imply that the person should keep the information confidential: there is an
implied duty of confidence
>From your letter it appears that UEA is claiming an exemption of the first kind, as you
cite a number of supposed confidentiality agreements that you do hold, which are
available at [4]http://www.cru.uea.ac.uk/cru/data/availability/ . In fact the great
majority of these are not clearly confidentiality agreements:

a) The 1994 FAX to the Met Office is simply a statement from Dr Hulme about the planned
use of the data; there is no reply as to the conditions under which the data is
provided.
b) The 1993 letter from DNMI is a limited request for confidentiality not a formal
agreement, and is almost certainly superseded by WMO Resolution 40. If UEA wishes to
claim exemption under this clause it must first establish with DNMI that an express duty
of confidentiality still applies.
c) The form in Spanish simply states that the data should only be used for the specified
purpose, and as no purpose was specified this cannot establish a duty of
confidentiality.
d) The web page is simply a statement by the Met Office of its own policies; this
provides no evidence whatsoever of any duties under which UEA might hold data. It
further notes that NERC data centres may make the data available under certain
circumstances, so there is no absolute duty of confidence.
e) The 1994 letter from Bahrain International Airport is a limited request for
confidentiality not a formal agreement, and is almost certainly superseded by WMO
Resolution 40. If UEA wishes to claim exemption under this clause it must first
establish with Bahrain International Airport that an express duty of confidentiality
still applies.
I understand that in the past UEA has refused to release the data I have requested and
related data because the request came from a person who was not an academic. I remind
you that "No regard may be had to the identity of the person who is requesting the
information nor to the purpose to which they will put the information."

I also remind
you that "When considering the balance of interests, public authorities must have regard
to the interests of the person to whom the duty of confidence is owed; the public
authority's own interests in non-disclosure are not relevant to the application of this
exemption." I further remind you that "If you receive a request for information which,
although it was confidential when it was obtained, was obtained a long time ago, you
should consider carefully whether the disclosure of that information would still
constitute an actionable breach of confidence within the meaning of section 41."
At best UEA has limited evidence for the existence of limited confidentiality agreements
covering part of the data I have requested. It is not clear to me that these documents
in any way establish an express duty of confidence. However, even if they do, the
responsibilities of UEA under Reg. 12(11) of the EIR are clear.
Regulation 12 (11) says: (11) Nothing in these Regulations shall authorise a refusal to
make available any environmental information contained in or otherwise held with other
information which is withheld by virtue of these Regulations unless it is not reasonably
capable of being separated from the other information for the purpose of making
available that information.
Thus UEA is certainly required to provide me with all the data I have requested with the
possible exception of data held under an express duty of confidence (for data withheld
it is required to establish that such an express duty of confidence does in fact
exist). Please note that if it is not possible to identify which data is covered by
supposed confidence agreements, then it is difficult to maintain that the release of
this data will breach such agreements.
I therefore appeal your decision, and reiterate my request for "a copy of any digital
version of the CRUTEM station data set that has been sent from CRU to Peter Webster
and/or any other person at Georgia Tech between January 1, 2007 and June 25, 2009".
--
Prof Jonathan A. Jones web page at [5]http://nmr.physics.ox.ac.uk
Oxford Centre for Quantum Computation and Brasenose College Oxford

"We've not deleted any emails or data here at CRU. I would never manipulate the data one bit - I would categorically deny that."

date: Wed Dec 3 13:31:06 2008 from: Phil Jones
subject: Re: FW: FOI_08-50 ; EIR_08-01 to: "Palmer Dave Mr"

Dave, Do I understand it correctly - if he doesn't pay the £10 we don't have to respond?

With the earlier FOI requests re David Holland, I wasted a part of a day deleting numerous emails and exchanges with almost all the skeptics. So I have virtually nothing. I even deleted the email that I inadvertently sent. There might be some bits of pieces of paper, but I'm not wasting my time going through these.

Cheers Phil

date: Mon, 24 Aug 2009 08:24:47 -0500
from: Harold Ambler
subject: Re: transparency
to: Phil Jones

Dear Phil,

Thank you for your prompt and thoughtful response.
...

2. I do not share the view that the days when amateurs contributed meaningfully to the
development of science have come to an end. If you have studied the history of science,
particularly that of your own great country, then you already know that non-academic,
frequently self-taught individuals have changed the scientific debate permanently in a
given field because of their own (frequently scoffed-at) work. Your somewhat condescending
position toward "non-scientists" is in keeping with Royal Society snobbery of the 19th
century, and I would ask you to consider the possibility that, like those of the society,
your efforts to keep the barbarians at the gate will be shown in the end to be a waste of
time.

3. You write, "Our ftp site has had some data deleted from it. It is a site we use when working with other scientists around the world. The datasets were not explained. It seemed easier to stop people wasting their time trying to determine what it was."

I admit that this does not seem as straightforward as, again, one might expect from a public servant.

The decision to delete data was made during a white-hot dispute with a little-liked and extremely dogged and intelligent statistician by the name of Steve McIntyre. Whether or not you view Mr. McIntyre as the kind of figure whom the Royal Society
fought to keep on the margins of scientific inquiry (or farther out than that), he is exactly such a figure.

If you wanted to "defeat" him in intellectual battle, as you naturally would, the best way
to do so is not to hide data and maintain that you are not hiding data.

The data should be restored to the website, ASAP. Mr. McIntyre should be allowed to "audit" your methodology. If your intellectual position is truly superior to his, then the "schooling" that you give to him in response will be of note to many.

Thank you for your consideration.

All the best, in all things,

Harold

"Phil, you must be very careful about deleting material, more particularly when you delete
it."

".. the pre-1930 temperatures are now very different, being much cooler(by > 0.5 degC for a 25-year low-pass mean) in CRUTEM3v than CRUTEM2v.
Previously they had been, on average, near or even above the 1961-1990
mean, now they're at -0.5 degC.

Is this a result of some homogenization work on New Zealand summer temp
data? Or just some random artefact of minor changes somewhere?

Phil, I had a look at figure 1 of Brohan et al. showing land station
> coverage and coloured dots for new/deleted/edited stations. All in
> New Zealand are black, implying no change from CRUTEM2! I'll check
> with programs again in case I screwed up, but fairly confident I
> didn't -- so its rather confusing as to why CRUTEM2 and 3 are so
> different over New Z. in summer. Tim

Tim,
Maybe you should see David if he can recall making any changes to land
stations over NZ - and if so when? There might have been changes
immedaiately after Jones and Moberg (2003) that Harry wasn't aware of.
Another thought is to check whether your programs work with CRUTEM2 as
of now. CRUTEM2 data are on the CRU site, go to the temp page and they
are near the bottom.

Cheers
Phil

Climate Science in the School is closely linked to Atmospheric Sciences and GeoSciences but permeates all parts of the School as the impacts of climate change on the earth system become a major focus for research activity. The core of this group is the world renowned Climatic Research Unit and ranges from reconstruction of past climates through to prediction of future climates. The new Tyndall Centre will further extend this research to consider strategies to minimise the impact of climate change on the UK.
...
The Climatic Research Unit undertakes pioneering research on the nature, predictability and impact of natural and anthropogenic climate change, maintaining a position as a world authority in the field. Five lead authors for the latest Intergovernmental Panel on Climate Change (IPCC) are drawn from the unit. The global land and marine surface temperature record maintained by the Unit is used extensively in climate change studies. The work of Jones and co-workers with this record can justifiably be claimed to have been central to the acceptance that humans are having a significant impact on climate.

Also, note that I've changed the way we smooth the series to preserve the late 20th
century trend, like we did in the Eos piece. I've always estimated the uncertainties a
bit more conservatively as described in text--so they're a bit expanded now. None of the
conclusions change, although the globe is actually a bit more anomalous in the late 20th
century when you preserve the late 20th century trend in the smoothing, so I've tweaked
the wording there just a bit...

date: Tue Jul 5 10:15:39 2005
from: Phil Jones

Dear Heinrich,

...The warming from 1915 to about 1940 is generally believed to be
due to a slight increase in solar output and a reduction in volcanism.

The recent
warming from 1975 is due to the build-up of greenhouse gases, as they
begin to dominate over sulphate aerosol releases.
The slight cooling from 1940 to 1975 is thought to be due to industrial
development (using lots of dirty coal) increasing aerosol emissions. WWII
is not thought to have had any impact.
On all the above there is natural variability which could have caused
some of the changes even without any external forcing (greenhouse gases,
aerosols, solar, volcanoes etc) as we have had warm and cold decades in
the past when there was little changes in external influences.
..

Det har vi sandeligt også haft helt frem til ca 2007.
(- eller rettere, vi har haft "a strong increase in solar output" frem til 2007.)

Passer ikke, Frank. Vildledende disinformation (igen)

vi har haft "a strong increase in solar output" frem til 2007

" At present
> the description of the global mean temperature record is for a warming
> of 0.6C during the 20th Century."

"Kevin,
Even without smoothing it is possible to get a trend of nearer 0.75 if the trend
starts around 1920 (especially if the cold year of 1917 is at the start). The
periods chosen for Table 3.2.2 had some justification, so we need to be a
little careful. "

Og her endnu længere horisont: