1. finance
2. accounting and auditing

Issue 26 ¦ May 2008
it
Why IT projects fail
1
In this issue
Austria
4
Country focus
Oman
12
Why Information Systems Projects Fail
Jordan
18
Why IT projects fail
Germany
26
Key issues for relying on External Consultants for Public Sector IT Projects
Sweden
30
Effective IT Governance:How to Get Good, Secure IT Services
Slovenia
36
Audit of IT system of the Tax administration of the Republic of Slovenia
Lithuania
38
Shall be blessed
Japan
42
Audit of Computer Systems used by the Japanese Government
China
CNAO’s Pre-audit investigation Guideline for IT Audit
46
Editorial
This edition covers a wide range of topics, and includes some further material based on
presentations at INTOSAI IT Audit Working Group’s Performance Audit Seminar held on Oman
in 2007 on the topic “Why IT Projects Fail”. This is a subject which we all return to time and
time again. Despite a huge amount of diagnosis, and lots of guidance on the cure, why do
failures still arise, and worse still why do the same problems arise over and over again?
Steve Doughty
It is interesting to note that, for example in the UK, Government has been computerising since
the 1950s. The delivery model was then largely in-house development, but over the years this
as moved through in-house but involving external consultants, contracting out, market testing
and outsourcing, Private Finance deals, and framework and strategic “partnerships”. But despite
all these different delivery models failures still happen. Common words that feature in reports
are words like “large, complex, inflexible, lack of user involvement, lack of training…”.
Of course, we do not have all the answers, but what we do know is that if best practice is adopted
and that a project’s scope is kept at “what we know we can do now” and not allowed to expand,
budgets and timescales are realistic etc., the chances of success are significantly higher.
In this, another bumper edition, we have contributions from Austria, China, Germany, Japan, Jordan,
Lithuania, Oman, Slovenia and Sweden. We are most grateful for these articles which demonstrate
the huge and varied amount of IT audit and related activity going on around the world.
Whilst on the subject of credit to our authors, I must apologise to Stephen Kateregga,
Assistant Director of Audit, and Ashok Ghosh, Consultant, from the Office of the Auditor
General of Uganda. We unfortunately omitted to credit them for their very comprehensive
article on their experience of IT Governance in SAIs in Edition 25 of intoIT.
So, on behalf of our readers, a big “thank you” to all our contributors. The journal only survives
because of your contributions, so please keep them coming. We are particularly keen to include
a range of short news items about what is happening in the IT audit world in YOUR country.
We happy to receive these at any time – please email them to intoit@nao.gsi.gov.uk . We are very
much aware of the difficulty in writing in English if that is not your first language, and are very happy
to provide help and guidance to make sure that your article gets published to the best advantage.
I am looking forward to hearing from you soon (yes, that means you!).
Steve Doughty
Editor
IntoIT is the journal of the INTOSAI Working Group on IT Audit. The journal is normally published
twice a year, and aims to provide an interesting mix of news, views and comments on the
audit of ICT and its use in Supreme Audit Institutions (SAIs). Material in the journal is not
copyrighted for members of INTOSAI. Articles from intoIT can be copied freely for distribution
within SAIs, reproduced in internal magazines and used on training courses. The Editor
welcomes unsolicited articles on relevant topics, preferably accompanied by a photograph
and short biography of the author, and short news items for inclusion in future issues.
The views expressed by contributors to this journal are not necessarily those of the editor or publisher.
Contributions should be sent to:
The Editor of intoIT
National Audit Office
151 Buckingham Palace Road
London SW1W 9SS
United Kingdom,
E-mail: intoit@nao.gsi.gov.uk, Web site: www.intosaiitaudit.org
Country Focus
Austria
Le
Leipzig
Bonn
B
Frankfurt
nkfurt Am M
Maain
Main
Lubin
Lu
Lod
Wrocla
aw
B
g
LUX.
Kra
ra
Nurnberg
urnberg
Pr
Prague
Karlsru
sruh
ruhe
uhe
Ostrava
O
Ost
t
CZECH
H
CZECH
Stuttgart
Stuttg
gart
art
B
Brno
sbourg
bou
SLOVAKIA
Munchen
Mun
Munchen
en
B
Bratislava
r
Vie
Vien
enna
Vienna
Zurich
Z
u
Bern
Ber
n
Innsbruc
bu k
Innsbruck
G
Gyo
Gy
Gyor
Gyor
Bud
SWITZERLAN
SW
WI
WITZERLAN
AN
ND
ND
Rhin
e
GIU
UM
U
AUSTRIA
R
AUSTRIA
Gra
G
raz
ra
Graz
A
SLOVENIA
Ljubljan
Ljubljana
naa
M ano
Mila
no Venezia
Turin
rrin
Genova
G
ova
enova
Rijeka
eka
Rijeka
naco
aco
co
s
seille
P
Zagreb
Zag
b
CROATI
OATIA
OA
IIA
CROATIA
Bolo
logna
lo
Bolo
l
Firenze
renze
HUNGARY
H
UNGARY
G
BOSNIA
BOSNIA
Sara
rajevo
raj
vo
Split
YUGOSL
U
AUSTRIA
Historical background
First traits of human settlement in Austria
can be traced back to the Middle Palaeolithic
era (Neanderthal). The most important
archaeological evidence of Palaeolithic
art was found in Lower Austria (Fanny of
Galgenberg near Krems, around 32,000 BC,
or Venus of Willendorf, around 25,000 BC).
In the Copper, Bronze and Iron Age,
people mainly focused on producing widely
used raw materials, building trade centres
and extensive trade routes. Iron Age culture
was dominated by the Celts: the Hallstatt
Culture even named the Early Iron Age.
From 15 BC onwards, the area of today’s
Austria, which then comprised the provinces
Noricum, Pannonia and Raetia, officially
became part of the Roman Empire with
important settlements such as Carnuntum
near Vienna (Vindobona), Virunum and
Magdalensberg near Klagenfurt.
After the slow decline of the Roman
Empire and the confusion of the Migration
Period (settlements by Goths, Slavs and
Avars) part of today’s territory was included
into the Carolingian Empire as marches.
The most important example of the art
of those times is the Tassilo Chalice at
Kremsmünster Abbey (around 780).
Already in the 9th century there was an
“Ostmark” (Ostarichi), which became the main
land of Austria in the 10th century. The first
record showing the name Austria dates from
996 where it is written as Ostarrîchi.
In 1278, Rudolf I, of the House of
Habsburg, secured the Austrian duchies
effectively for the House of Habsburg for the
next 650 years The Habsburgs ruled almost
constantly as Holy Roman Emperors from
the mid 15th century onwards. In addition
to the hereditary lands of Lower Austria and
Styria, they soon acquired Carinthia, Tyrol
and Carniola. Salzburg only became part of
Austria in 1816.
Since the 15th century, Austria was
being threatened by the Ottoman Empire.
In 1529 and 1683 the Turks besieged Vienna
– although without success. Prince Eugen’s
final defeat of the Turks at the end of the
17th century marked the beginning of an
unprecedented golden age in Austrian
culture. Austria and especially Vienna
became the hub of Baroque architecture and
painting. Artists such as Fischer von Erlach,
Lukas von Hildebrandt and Jacob Prandtauer
influenced profane and sacred Baroque
architecture in Austria and Central Europe.
During their reign, Maria Theresia (1740
to 1780) and her son Joseph I initiated
fundamental state reforms (compulsory
school attendance, absolute discharge,
administrative reforms).
Emperor Francis II abdicated as Holy
Roman Emperor in 1806 under pressure from
Napoleon. Already two years before, in 1804,
the Empire of Austria had been founded on
the remnant of the Holy Roman Empire and
existed until 1867.
In the late 18th and early 19th century,
the Classical period of Western music
flourished in Vienna. Best-known composers
of this time were Joseph Haydn, Wolfgang
Amadeus Mozart, Ludwig van Beethoven
and Franz Schubert.
The Austrian empire of 1804 and the dual
monarchy Austria-Hungary (1867 to 1918)
were multi-ethnic empires belonging to the
German Federation. Like Prussia and Russia,
Austria remained an absolutistic state, which
was dominated by Minister Metternich’s
police. This time of domestic retreat,
of “Biedermeier”, bore fruit in painting
(Ferdinand Georg Waldmüller), literature
(Franz Grillparzer, Johann Nestroy) and music
(Franz Schubert).
In 1848, the peoples of the monarchy
fighting for democracy and independence
started the first revolution. They were
defeated and Franz Joseph I was enthroned.
Military defeats against Italy and Germany
weakened the Habsburgs and led to
profound political reforms. In the 19th
century, Austria was also industrialized.
But it was also a period of great cultural
achievements. The historical buildings on
Ringstrasse in Vienna (opera, parliament,
university, etc.) were constructed and
Venus of Willendorf
the composers Johannes Brahms and
Anton Bruckner worked in Vienna along
with Johann Strauss and Gustav Mahler.
At the beginning of the 20th century
Jugendstil art (Otto Wagner) evolved
around a group of artists who objected to
prevailing conservatism. Representatives
of the “Second Viennese School (Arnold
Schönberg, Alban Berg, Anton Webern)
set the direction music developed in the
20th century. This period surrounding the
downfall and end of the Austrian monarchy
was best described in the literary works of
Stefan Zweig and Karl Kraus.
In the early 20th century Vienna was
also a scientific centre. World-renowned
researchers in natural sciences (physics,
medicine), philosophy or economics worked
there at the time.
The assassination of Archduke Franz
Ferdinand in Sarajevo in 1914 by a Serbian
nationalist was the immediate cause for
the outbreak of World War I, leading to the
downfall and the end of the monarchy.
Emperor Franz Joseph died in 1916 and
his successor Karl I had to leave Austria, when
the first republic was founded in 1918.
In the post war years, hyperinflation
shook the young republic of Austria and
could only be ended by introducing a new
currency. This led to modest economic
upswing, but in 1933 one third of the
workforce was still unemployed. There was
political tension between the opposing
parties, which led to turmoil and civil war.
Austria’s bad economic state in the 1930s
encouraged the occupation by the German
Reich in 1938. After World War II, Austria
became an independent state again, but was
also divided into occupation zones.
Only in 1955, Austria regained full
independence by concluding the Austrian
State Treaty with the Four Occupying Powers.
Austria became a member of the United
Nations in 1955, on 1st January 1995 it joined
the European Union and in 2002, the EU’s
common currency, the Euro, was introduced.
5
Geographical background
60 per cent of Austria’s territory is located
in the Eastern Alps. Therefore, Austria is
often referred to as “Alpine Republic” in
the media. There are larger plains along
the Danube valley, in the Vienna Basin,
in southern Styria and Burgenland.
Austria’s highest mountain,
Grossglockner (3,798 m) is located in the
mountain range of Hohe Tauern between
Carinthia and Tyrol. The second highest
peak is Wildspitze (3,768 m), followed
by Grossvenediger (3,674 m). Parts of
the alpine region of Carinthia, Tyrol and
Salzburg form the national park Hohe
Tauern. This protected area comprises about
1,836 sq km. The highest alpine pass is
Grossglocknerstrasse with an elevation of
2,504 m at Hochtor, its highest point.
Lake Constance and Lake Neusiedl are
the largest lakes, parts of both being on
Austrian territory. The lakes of Carinthia
(lakes Ossiach, Millstatt and Weissensee), the
lakes of Salzkammergut (Wolfgangsee and
Mondsee) are of special importance
for tourism.
Natural riverside areas can be found in
all of Austria. The most important rivers are
the Danube in Upper and Lower Austria and
in Vienna, the Drau in Carinthia, the Inn in
Tyrol, the Salzach in Salzburg, the Rhine in
Vorarlberg and the Mur and Mürz in Styria.
Austria lies in the temperate climate
zone with oceanic and continental features.
Generally, we talk about Central European
transitional climate or alpine climate in the
alpine region, where the winters are cooler
due to higher elevations.
2005
2006
Growth in GDP (“securing
growth opportunities”)
+ 2.0%
+ 3.3%
Inflation rate (“sufficiently
stable currency”)
+ 2.3%
+ 1.5%
Unemployment rate
+ 7.3%
+ 6.8%
5.2%
4.8%
Employed
(“high employment”)
+ 1.0%
+ 1.7%
Balance on current account (absolute)
+ Є5.16 billion
+ Є8.22 billion
Balance on current account (in relation
to GDP) (“foreign trade balance”)
+ 2.1%
+ 3.2%
Unemployment rate
according to EUROSTAT
Source: AMS; WIFO; Statistik Austria
The main reasons for the high income
from tourism lie in undisturbed mountain
and lake regions, the high number of cultural
institutions, good infrastructure, the central
position in Europe, modern accommodation
and the high training level of staff.
Tourism concentrates on three branches:
¬
City tourism concentrates mainly on
Vienna and the Länder capitals. Tourists
visit Austria the whole year round for
cultural trips, adventure holidays or to
attend conferences.
¬
Winter tourism is mostly concentrated
on the alpine regions. Besides skiing
holidays, health trips to the thermal
regions get ever more attractive.
In glacier regions skiing is possible all
year round.
¬
In summer, tourists in Austria prefer
active holidays and holidays at the lakes.
Active tourists go cycling or paragliding
and, ever more frequently, hiking and
mountain climbing. In the past years
summer and winter resorts have been
transformed to all-year resorts to secure
all-year occupancy.
Economic background
Development of the gross domestic
product (nominal):
The indicators for macroeconomic
equilibrium (high employment, sufficiently
stable currency, securing growth
opportunities, foreign trade balance) show
the following development as compared to
the past years:
GDP (billion €)
2004
237
2005
245
More and more tourists stay in better
hotels. There is a trend towards shorter
holidays, which are booked later as the
tourists wait for snow reports in winter
and for sunny periods in summer.
2006
258
Politics
Tourism
Tourism is one of the most important
business sectors in Austria. 120 million
overnight stays were reported in Austria
for 2006. They amounted to an income
of 13.3 billion Euro from non-resident
visitors. The tourism net currency
receipts accounted for 5.9 billion Euro.
6
Austria is a democratic republic. Its law
emanates from the people. Austria is
a federal republic with nine Länder:
Burgenland, Carinthia, Lower Austria,
Upper Austria, Salzburg, Styria, Tyrol,
Vorarlberg and Vienna (federal capital). It
has a federal constitution. Since 1955 Austria
has been member of the United Nations;
in 1995 it joined the European Union. Its
currency is the Euro within the framework
of economic and monetary union.
The federal president is the highest
representative of the state. He or she is
elected for six years and can be re-elected
once. The government consists of the federal
chancellor and the ministers. The president
nominates the chancellor and the ministers
upon suggestion of the chancellor.
There are two legislative bodies in
Austria. The National Council is the dominant
chamber of the Austrian legislation and is
elected by the people based on proportional
representation. The National Assembly is
elected for five years, if the National Council
or the federal president and the government
do not shorten the legislative term.
The number of members of the Federal
Council is based on the population of the
länder. The Federal Council has an absolute
right of veto, when the competence of the
Austrian Länder is limited by constitutional
laws. In all other cases it has suspensive veto,
which can be reversed by an inertia decision
of the National Council.
The Austrian Court of Audit
The Austrian Court of Audit (ACA) is Austria’s
independent supreme audit institution
and has especially been established for the
audit of the federal, länder and local levels
of government. In addition to auditing the
financial management and consultation
services based thereupon – its most
important strategic function – it also renders
further services with major importance
for the good governance of the state and
performs special notarial functions.
The Austrian Court of Audit audits
the accounts and the financial
management of the Austrian state
for the National Council, the Länder
parliaments and the municipal councils.
Some facts about Austria
Inhabitants
8.3 million
Area
83,871 sq km – somewhat smaller than the
US state of Maine; somewhat smaller than
Portugal, about as large as Japan’s second
largest island Hokkaido; somewhat smaller
than French-Guiana.
Religions
5.9 million Roman-Catholics (73.6%)
380,000 Protestants (Augsburg Confession,
Helvetic Confession) (4.7%)
340,000 Moslems (4.2%)
180,000 Orthodox (2.2%)
8,140 Israelites
0.96 million without religious confession
Language
German
Regional languages
Croatian
Slovenian
Hungarian
Recognised
Sign language
As an independent body of the National
Council, the Austrian Court of Audit audits all
financial operations of the federal government,
enterprises in the ownership of the federal
government and other legal entities defined
by law. The audits of the Austrian Court of
Audit are based on the principles of economy,
efficiency and effectiveness as well as on the
correctness of the accounting compliance with
existing regulations.
When auditing the financial operations
of the länder, municipalities and municipal
associations, the Austrian Court of Audit
acts as independent audit institution of the
länder parliaments. It audits the financial
management of the respective land,
enterprises of the land government in the
ownership of a land alone or together with
other entities defined by law as well as the
financial management of municipalities
with at least 20,000 inhabitants (including
enterprises the municipality controls).
The Austrian Court of Audit also audits
the financial management of the social
insurance institutions and other legal entities
defined by law.
Following the audit mandate laid down in
the Austrian Constitution, the Austrian Court
of Audit is independent of the government
and directly reports to legislative assemblies
(National Council and länder parliaments).
The Austrian Court of Audit reports to the
National Council, the länder parliaments
and the municipal councils on its activities
and individual findings. The reports to the
legislative assemblies have to be published
after being submitted to the National
Council, the respective land parliament or
municipal council.
Objectives and mandate
Mandate
The Austrian Court of Audit fulfils its
mandate – as laid down in the federal
constitution – directly towards the National
Council, the Länder parliaments as well as
decision-makers in politics, administration
and the economy. Even if it acts on a request
or motion, the Austrian Court of Audit
remains independent and is not bound by
any instructions.
Objectives
As its primary objective, the Austrian Court
of Audit aims at the most effective use of
public funds, i.e. cost reductions on the one
hand, and increased benefits from the use
of public funds on the other hand. It checks
whether public funds are raised and used in
a lawful manner, as well as in a cost-efficient,
economical and effective way and according
to the principles of sustainable development.
Thus the Austrian Court of Audit fulfils
its mandate as laid down in the federal
constitution, for the optimisation of income
and expenditure. This strategic objective
is also in line with the Austrian Court of
Audit’s aim to increase the efficiency and
effectiveness of public auditing.
Organisation of the Austrian
Court of Audit
The Austrian Court of Audit is headed
by the President, who is elected by the
National Council upon proposal of its Main
Committee for a single term of office of
twelve years. His deputy is the highestranking official of the Austrian Court of Audit.
The President has the same
responsibilities as members of the
federal government.
He is entitled to participate in the
debates of the National Council as well as
of its committees and sub-committees
dealing with ACA reports, federal financial
statements and pertinent sections of the
Federal Finance Bill.
The Austrian Court of Audit is headed by
the President and organised monocratically.
The President is responsible for the decisions
taken. The new organisation centralises its
services in five Directorates-General and
35 specialised departments. To support
selective as well as crosscutting work
and audits, related tasks and audit areas
have systematically been assigned to the
respective Directorate-General.
7
President
Cabinet
Directorate General 1
Directorate General 2
Directorate General 3
Directorate General 4
Directorate General 5
Department S1-1
Department S2-1
Department S3-1
Department S4-1
Department S5-1
Budget and infrastructure
Strategic planning,
controlling
Editing of reports
IT affairs
Communication and
parliamentary relations
Department S1-2
Department S2-2
Department S3-2
Department S4-2
Department S5-2
Education
Health
International affairs,
INTOSAI General
Secretariat
General legal and
economic issues
Human resources,
administration and
development
Department S1-3
Department S2-3
Department S3-3
Department S4-3
Department S5-3
Research
Hospitals
EU financial management
Federal financial
statements, national
budget, survey of public
sector incomes
Knowledge management
Department S1-4
Department S2-4
Department S3-4
Department S4-4
Department S5-4
Science
Social affairs
Waterway and
air transport
telecommunications
Banking, debt
management
Federal administration
Department S1-5
Department S2-5
Department S3-5
Department S4-5
Department S5-5
Culture, art and media
Energy
Building and construction
Economic affairs,
competition
Länder administration
Department S1-6
Department S2-6
Department S3-6
Department S4-6
Department S5-6
Foreign affairs and
defence
Comprehensive
environmental protection
Land transport
Labour market affairs
Local governments,
municipal associations
Department S1-7
Department S2-7
Department S3-7
Department S4-7
Department S5-7
Judicial administration
and domestic affairs
Urban and regional
planning
Fiscal administration
Real estate, property
administration
Organisation, staffing, IT
systems
By handwritten letter of 23 December
1761 Empress Maria Theresia establishes
the predecessor of the present Court
of Audit, the “Accounting Chamber”,
assigning it the duty “to scrutinise
all accounts and to point out all
shortcomings discovered in matters
relating to public finance and, in
particular, public spending”.
The Government Audit Act leads to the
reorganisation of the supreme audit
institution, which is now directly and
exclusively accountable to the National
Assembly. At the same time the mandate
and the authority of the government audit
institution are enlarged.
Its functions, in particular, include:
1. expressing an opinion with suspensory
effect on all important financial
matters until a decision is made by
the Empress, i.e. exercising the right
to preventive control;
The Supreme Court of
Audit of the monarchy
is incorporated into the
institutional framework
of the newly established
Republic. It is answerable to
the State Council.
2. making recommendations for
improvements to accounting
methods;
3. assuming the direction and guidance
of all accounting agencies.
1761
8
Staff Units
...
1918
The Court of Audit is now entitled to audit
all financial operations of the government
as well as government debts. The audit
mandate is moreover expanded to
foundations, funds, institutions and entities
in which the government holds a financial
interest. The Court of Audit is charged with
verifying compliance of the administration
of public funds with existing laws and
regulations. It is also tasked with a new
field of competence, namely assessing the
economic efficiency and expediency of
financial operations.
1919
The Federal Constitutional Act of
1 October 1920 dedicates an entire
chapter to government audit, thus
providing its basic constitutional
framework. The Austrian Court of
Audit is directly subordinated to the
National Council (first chamber of
parliament). The President of the
Austrian Court of Audit is elected by
the National Council upon nomination
by the Standing Committee and can
be removed from office by resolution
of the National Council.
At the level of the Austrian Länder it is
within the discretion of the respective
constitutional assemblies to confer
upon the Austrian Court of Audit the
same audit authority exercised by
the Court at the federal level
(optional jurisdiction).
1920
...
President of the Austrian Court of
Audit and of INTOSAI
¬
Member of the Supervisory Board
of ÖBB-Infrastruktur-Betriebs AG
(railway infrastructure company)
Education
¬
Managing Director of
ÖBB-Immobilienmanagement
GmbH (real estate management
of the Federal Railways)
Compulsory school, secondary
higher school, A-levels in 1975.
Law studies at Vienna University, Austria,
graduation in 1981 with a doctoral degree.
Professional experience
Government administration
Different management functions in the
federal financial administration (e.g.
staffing and general policy matters) and
in the provincial administration (e.g.
managing function in the cabinet of
the province governor of Carinthia).
Parliament
Executive director of a political group in
parliament; responsible for coordinating
activities between the government and
parliament; promotion of intra-parliamentary
cooperation between the representatives
of the different political parties; direct
cooperation with the federal ministries.
Private sector
Different managing functions in the private
sector, e.g.:
¬
Member of the Board of Directors of
Eisenbahn-Hochleistungsstrecken AG
(railway construction company)
¬
Member of the Board of Directors of ÖBBHolding AG (railway holding company)
1 July 2004: President of the Austrian Court
of Audit and Secretary General of the
International Organization of Supreme Audit
Institutions (INTOSAI)
¬
Professional Standards Committee (PSC)
¬
Subcommittee on Internal Control
Standards
¬
Capacity Building Committee (CBC)
¬
Sub-committee 3: Promote best practices
and quality assurance through voluntary
peer reviews
¬
Working Group on IT Audit
¬
Working Group on Environmental
Auditing
¬
Working Group on Privatisation,
Economic Regulation and Public-Private
Partnerships
¬
Working Group on Programme
Evaluation
¬
Working Group on Accountability for and
Audit of Disaster-related aid
General Secretariat of INTOSAI
The General Secretariat is located at the
Austrian Court of Audit in Vienna. Under
the direction of the Secretary General
the tasks of the Secretariat include the
management of the accounts of that
worldwide umbrella organisation.
According to its statutes, INTOSAI
consists of the Congress, the Governing
Board, the General Secretariat and various
committees and working groups. In 1977
the IXth Congress in Lima, Peru, adopted the
“Lima Declaration of Guidelines on Auditing
Precepts”.
In line with INTOSAI’s motto “experientia
mutua omnibus prodest” (Shared experience
benefits all) the General Secretariat keeps in
contact with the more than 180 members in
the time between congresses and organises
seminars, expert meetings and other events.
As Secretary General:
¬
ex-officio member of the Finance and
Administration Committee
¬
observer in the PSC steering committee
¬
chair of the INTOSAI task force on
Communication Strategy founded at the
XIXth INCOSAI
The provisional constitution provides for a
State Court of Audit “to perform the financial
audits and monitor the financial operations
of the federal and Länder governments, the
local authorities in communities with more
than 20,000 inhabitants and their enterprises,
institutions and other legal entities”.
The mandate of the Austrian
Court of Audit is transferred
to the Court of Audit of the
German Reich, which establishes
a branch office in Vienna.
1939
ACA is member in the following committees,
working groups and task forces:
...
1945
The Fifth Chapter of the
Constitution is amended
and the new Court of
Audit Act is passed.
...
1948
9
The European Commission chose
20 basic public services and worked out
an assessment system for the survey.
The EU-wide survey resulted in a score for
the online sophistication of 75 per cent.
Full availability online has reached almost
50 per cent.
In both indicators, Austria heads the
results in providing eServices for its citizens.
95 per cent of the administrative tasks
were available online, 83 per cent could be
completed online.
Audit principles
The Austrian Court of Audit audits the
economy, efficiency and effectiveness
of the public administration and of
public enterprises on the basis of
regularity and legality in the interest
of sustainable development.
After auditing the legality of financial
management, regularity is audited.
Furthermore, savings potentials are
defined for:
¬
efficiency: optimum relation between
input and output (target-performancecomparison);
Auditing information technology
¬
economy: minimum expense for the task
to be performed;
¬
effectiveness: high degree of target
achievement;
and recommendations are issued.
E-Government services in the
Austrian administration
The Austrian administration was ranked top
in the EU’s e-government benchmarking
exercise of 2006 for “Online Availability of
Public Services: How is Europe Progressing1“.
Two indicators were used to monitor the
eEurope action plan: online sophistication
of basic public services available and
public services fully available online.
These indicators were defined in 2000
and evaluated in 2006. Assessed were the
online sophistication of basic public services
in the EU member states, in Norway, Iceland
and Switzerland.
The 5th International Congress of Supreme
Audit Institutions (INTOSAI) resolves to set
up its permanent international secretariat at
the Austrian Court of Audit.
1965
10
...
ACA also performs IT audits. An ACA
team deals with essential aspects such
as legislature in e-government and
its implementation. This team also
completes performance audits, which are
performed at all levels and in all areas.
To cover all aspects of this task, the team
consists of technicians and lawyers.
IT is an essential element for modernising
and optimising the administration. Therefore
ACA tries to consult auditees and provide
them with information on new technological
developments and processes.
In general, procurement, technology,
accounting and fulfilment of the contracted
services are audited in IT projects. Apart
from auditing IT implementation projects
and new developments, special focus is laid
on crosscutting audits of the administrative
institutions. ACA compares whether these
institutions want to implement similar
technological innovations and whether
synergy effects can be identified.
Essential elements in auditing are
fulfilling the functionalities and the scope of
available functionalities and infrastructure.
ACA stated that today’s technological
opportunities lead to functionalities, which
are too comprehensive for the users.
IT audits
Health insurance card in Austria;
e-card
The electronic health insurance card
(dubbed “e-card”) is an electronic ID for
using medical services from health insurance
in Austria; on the reverse side, it incorporates
a printed image of the European Health
Insurance form. As an additional feature,
the e-card may also be used as a Citizen
Card after a no-charge certification process.
As of January 2006, the e-card replaced
the paper-based health-insurance voucher
(“Krankenschein”) in Austria.
The card holder data are stored and
administered centrally in two parallel
computing centres and made available
online depending on the application and
the access rights. Data are transmitted after
encryption and signature; the signature
is activated without a PIN code when the
e-card is inserted in the card reader. The
e-card system was designed as an online
system and can be expanded for future
uses, such as for electronic prescriptions, or
patients’ electronic case histories.
The Austrian Constitutional Convention proposes
to enlarge the scope of responsibilities of the
Austrian Court of Audit and to expand its audit
authority to encompass municipalities with less
than 20,000 inhabitants, stock corporations in
which the public sector holds a stake of 25 per cent
or more and EU direct aid. A special committee
is set up in the National Council to commence
preliminary deliberations on the report submitted
by the Austrian Constitutional Convention.
The deputisation rules regarding the
representation of the President are re-drafted.
The function of Vice President is abolished. If
the President is prevented from discharging
his or her duties, the President is to be
represented by the most senior civil servant of
the Austrian Court of Audit.
1994
...
2005
Government budgeting NEW
Reorganisation of IT–based government
budgeting on the basis of managerial
standard software was introduced in 1998
and successfully completed in May 2004.
Central idea of Government budgeting
NEW was the appropriation of budget law
data on expenditure and income directly
to the relevant department in the ministry.
Bookkeeping should audit the data and
non-cash performance. By transferring
the tasks completed by bookkeeping
to the ministry, bookkeepers in the
ministries can be reduced by one half.
Electronic files in public
administration
In public administration electronic files
substitute paper files and are considered
original files. Each print-out of an electronic
file is a copy of the original. In administration
the electronic file substitutes paper and
all related procedures until filing.
Eleven federal ministries and BHAG
(Austrian federal bookkeeping agency) have
used an electronic filing system acquired
from the same producer in one procurement
exercise in January 2005. Only the Ministry
of Defence has used its own electronic filing
system since 2002. |
Contact
Johann Vilanek
(vilanek@rechnungshof.gv.at)
1
http://ec.europa.eu/information_society/eeurope/
i2010/docs/benchmarking/online_availability_
2006.pdf.
Dr Josef Moser
Dr Moser graduated in 1981 from the University of Vienna
with a doctoral degree in Law. After a career in the public
service in Carinthia, from 1992 to 2003 he was executive
director of a parliamentary group in the Austrian parliament;
and from 2002 also responsible for coordinating activities
between the coalition parties and the federal ministries.
Afterwards he had different managing functions in the ÖBB
(railway company). Since 2004 he has been President of the
Austrian Court of Audit and Secretary General of INTOSAI.
11
f
a
Str. of
Horm
r uz
R N
RAI
T
TA
Om an
h
Musc
cat
U. A. E.
Dhab
abii
Dhabi
I A
Al Khal
a uf
O M AN
Al Ghayd an
Arab ian Se
Sal a
all ah
N
Al Mu
M kal l a
OMAN
By Awatif Amin Qassim,
Specialist – Information
Technology Department
State Audit Institution – Oman
awatif@sai.gov.om
12
Why information
systems projects
fail: Guidelines
for Successful
Projects
Introduction
Information and communication technology
(ICT) plays an essential part in our life and the
way we react with the external environment.
Information systems are one of the ICT
elements which shape our daily tasks by
introducing value and quality to our daily
activities. Technologies are emerging fields
and are rapidly changing and the changes
are moving around the globe in developed
and developing countries.
Information systems are the core of
today’s emerging businesses. Billions of
dollars are exchanged on daily basis based
on automated systems and information
technology. It is essential that information
system projects are properly scoped and
implemented successfully. According to
Gheorghiu, A. (2006), a survey showed
that around 70-80% of all information
technology and information systems
fail. Despite best practice and defined
procedures and methodology applied
in project management, as well as the
development and the advancement in the
project management field, the world is
still experiencing failures in implementing
information system based projects,
especially in developing countries in the
Middle East.
The gravity of information systems is
increasing day by day around the globe,
but at the same time the failure rate of
information systems projects is still high.
Why Systems or Information
Technology Projects fail
Different research and studies, regarding
information systems or information
technology project failure show the
highest risk factors that were behind the
project failure. The world’s statistics always
publish failure rate in general, which clearly
can prove for business and information
technology executives that there is failure at
IS projects regardless of whether it is high or
low for (IS) or (IT) projects. The key objective
of all the research and studies is information
and communications technology awareness
which can reduce or resolve failure rate
for a project by using the accurate and
professional techniques.
Different types of existing surveys results
published by IT Cortex providing statistical
information regarding the rate of failure in
IS or IT projects. Following are lists of the
existing surveys:
1. (2001) The Robins-Gioia survey.
2. (2001) The Conference Board survey.
3. (1997) The KPMG Canada survey.
4. (1995) The Chaos report.
5. (1995)The OASIG survey.
All Cortex statistics generally agreed on
the below points regarding the failure
at information systems and information
technology project:
¬
Unsuccessful IT projects are more likely
than successful projects.
¬
Nearly 20% of IT projects are satisfactory.
¬
Failure rates are much more likely in case
of large size IT projects compared to small
and medium size projects.
Information Systems and
Information Technology Project
Common Failure factors
Information systems projects always and
everywhere around the globe have a
reputation for failure, i.e. unused, partially
used, cancelled and many other factors. Each
project differs from another even if it is for
the same system because each project has
its own requirements, project management,
users, organisation culture, team skills and
knowledge, and many other aspects that are
linked directly to the organisation and not to
the project itself.
Different research studies have been
made which describe and summarise the
most common failure factors in IS projects.
Most of the results show similar failure
factors but each factor can have different
priorities which link to the project and the
14
organisation itself. Moreover, the project
and organisation always have a strong
relationship with each other which can shape
the final outcome of the project in terms
of failure or success. According to Dorsey
(2000), in all the studies that have been
done till now regarding Information systems,
failure or success have highlighted top
management support as a critical success
factor in any project. Any project without full
commitment from the top management, in
case of problems can collapse at any time
during the project life cycle.
One of the researches listed risk factors
ranking wise. ComputerWeekly.com joined
forces with Oxford University to carry
out a research into the state of IT project
management in the United Kingdom. The
research was led by Sauer, (2003) fellow for
information management at Templeton
College, and sponsored by the French
Thornton partnership. The aim of the
research was to help Information technology
and business executives create realistic
expectations for Information technology
projects and improve the performance of
project management, besides developing
the skills required for project management.
The most common risk factors ranking
wise were one of the outcomes of the
research. Table 1 opposite lists ranking wise
risk factors.
Although there is a high rate of
information systems project failure there
are ways of enhancement and areas of
improvements. Different books, research
and studies give clear improvement factors
that can help avoid failure in IS projects.
The improvements factors were published
after detailed study and investigation of
different kinds of IS projects among multiple
industries around the globe. The main aim
of all the existing improvement factors
is to reduce or resolve the failure rate at
IS projects. Moreover, they help the top
management and project managers to use
standard best practices and move towards a
technology world with minimum risk factors.
One of the research have clearly defined
improvement factors regarding information
systems project, Table 2 lists the factors.
The CHAOS study, which was conducted
by Johnson, et al (2000) has defined a recipe
for success as a CHAOS 10. Moreover, they
have clearly explained that no project
requires the entire 10 recipe ingredients for
success, but the more factors present in a
project, the more value can be added to the
project. Table 3 lists the CHAOS 10 recipe
ingredients for success. Each success factor
has been weighted according to its influence
on the project’s success. The more success
rate, the lower project risk.
Table 1: Risk ranking
Rank ing and risk
1.
Lack of top management
commitment
11. Shortage of knowledge/
skills in the project team
2.
Misunderstanding of scope/
objectives/requirements
12. Improper definition of roles
and responsibilities
3.
Lack of client/end-user
commitment/involvement
13. Artificial deadlines
4.
Changing scope/objectives
5.
Poor planning/estimation
15. New or radically business
process/task
6.
Inadequate project
management
16. Employment of new
technology
7.
Failure to manage end-users
expectations
17. Poor control against target
8.
Conflict among stakeholders
18. Number of organisational
units involved
9.
Change is senior
management ownership
19. Lack of effective
methodologies
10. Lack of adequate
change control
14. Specification not frozen
20. Staff turnover
21. Multiple vendors
Table 2: Ranking wise improvement factors
Ranking and factor
I.
A. Greater top management
support
Alignment of IT project
initiatives to business strategy
J.
Greater understanding of
project management on the
part of top management,
project boards and clients
B. More commitment from users
C. More power and decisions
making authority
D. Greater financial control and
flexibility
E.
Greater Control over staff
resources
F.
Commitment to requirements
and scope once specified
K. Greater realism in setting
targets. Several respondents
railed against imposed rather
than planned targets and
deadlines
L.
Establishment of a supportive
project/programme office.
G. More project management
training
H. Commitment to a stable project
management method
Table 3: CHAOS 10 – Recipe for Success
1.
Executive Support
18
2. User Involvement
16
3.
14
Experienced project manager
4.
Clear business objectives
12
5.
Minimised scope
10
6. Standard software infrastructure
8
7.
6
Firm basic requirements
8. Formal methodology
6
9.
5
Reliable estimates
10. Other criteria
5
15
Project Guidelines
The guidelines have developed by the author
after thorough research and investigation
into the information systems project failure
issue and the aim of the concern guidelines
was to resolve or reduce project failure rate
by following the accurate guidelines in small
or medium size IS projects. Twenty project
guidelines have been developed for the
three essential project stages, namely:
Table 4: Twenty project summary guidelines
Guideline Number
Before starting
the project
1. Prior to selecting a project
1
Analyse the organisation environment using
standard tools such as SWOT or PEST
2
Align Business with ICT Strategy
3
Ensure management buy-in
4
Ensure adequate project resources
5
Ensure project team have the required
skills and knowledge to run the project
6
Clearly define scope, objectives
and requirements
7
Break project down into
manageable components
8
Construct the project’s product to be
flexible and open to future change
9
Make use of previous experience
10
Establish clear criteria for supplier selection
11
Carry out detailed costing and
establish a feasible project budget
12
Maintain communication at all levels
13
Boost awareness inside the
organisation of the project
14
Adopt a good project management strategy
15
Create risk plan and monitor it
16
Establish timetable to give users enough
knowledge to accept new system
17
Establish documentation standards
and backup strategy
2. During the project
3. After the project execution
In reality it is not necessary for an
organisation to follow all the concerned
guidelines but to understand the
standard steps which can be followed
in a IS project. These guidelines
can be used as a best practice.
Each guideline is developed to keep a
project on the right track and minimise the
risk before, during and after the project.
The first and second project stages are
the most critical which require focus
and clear understanding not only from a
project manager but more importantly top
management.
Top management support is essential at
all the stages of a project before, during, after
the implementation.
Finally, the guidelines are not
developed exclusively for information
systems specialists, project managers and
technical people. Their aim is to cover and
help the entire range starting from top
management through to the ordinary users
in an organisation. The whole plan is to
understand what is required for a successful
information systems project.
Table 4, lists the twenty project
guidelines summary for all the three
stages before, during, and after project
implementation. Guidelines 3,12, and 13 are
repeated at more than one stage.
During the
project
Repeated Guidelines 3,12 and 13
After the
project
implementation
3
Ensure management buy-in
12
Communication at all levels is essential
13
Boost awareness inside the
organisation of the project
18
Periodic reviews once project is live
19
Consider ongoing user training
20
Establish a project knowledge base
Repeated Guidelines 3 and 12
16
3
Ensure management buy-in
12
Communication at all levels is essential
Messages from leaders and
professionals regarding the article
The culture 50 years ago in the world
was totally different from that of today,
which means it is still changing. So we
have to keep abreast of the changes and
receive what is best from the culture but
at the same time refrain from what is
not useful. The most important thing is
that we have to follow a new culture, the
culture of technology and information
H.M.Qaboos Bin Said (2007)
“Information technology is not a magic
formula that is going to solve all our
problems. But it is a powerful force that
can and must be harnessed to our global
mission of peace and development”
Kofi Annan (2003)
Awatif Amin Qassim
Awatif is a Specialist in the Information
Technology Department at the State
Audit Institution Sultanate of Oman.
She studied for her MSc in Information
Systems at Kingston University, London
Nicosia
CYPRUS
Eu
ph
rat
es
SYRIA
Beirut
e a
D
s
Damascus
LEBANON
Po r t
Said
Alexandria
Tel
Aviv
ISRAEL
Amm n
Amma
Amman
R.
I R A Q
ad Sea
Se
ea
Dead
phrate
RDAN
JORDAN
ro
Ben
S uez
ef
S A U D I
El-M
A R A B I A
R.
le
Ni
Al Manam
E G Y P T
Aswa n
Medina
Riyadh
Why IT
projects
fail
Mec a
Po r t S u
JORDAN
Research continually shows
that many information
technology (IT) projects all
over the world have
difficulties with completion
on time or on budget or
on scope. In fact many are
cancelled before completion
or not implemented.
Project success is affected by
many factors such as project
team, suppliers, customers
and stakeholders; the truth
is that they can all provide a
source of failure.
There are many different
reasons for the failure of IT
projects but the most common
reasons are rooted in the
project management process
itself, this paper covers the key
reasons of IT projects failures.
18
IT Projects fail when they do not meet one or
more of the following criteria for success:
¬
delivered on time,
¬
on or under budget,
¬
satisfies user requirements.
Only a few projects achieve all three.
So what are the key factors for IT project
failure? Organisations and individuals have
studied a number of projects, successful
as well as failed ones, and some common
factors emerged. A number of these
factors are involved in any particular project
failure and they interact with each other.
Here then are some of the most important
reasons for failure.
Lack of a project methodology
The project methodology or project
lifecycle describes the approach that will
be taken to carry out a project. Lack of a
project methodology will force the project
manager to make on-the-fly decisions,
based more on gut reactions than factual
and objective analysis.
Projects should follow a well thought-out
route to avoid going in circles, getting lost,
and hitting countless roadblocks. Taking an
unstructured approach is a risk that will lead
to unstable results because things rarely fall
into place by themselves.
Methodologies vary greatly from project
to project, taking into account environmental
factors and project specifics. And, of course,
the methodology is relative to the size of
and the complexity the project. The bigger
the project, the more important it is to have
a methodology. But regardless of size, every
project methodology must address three
core issues: planning, development and
implementation. By following a pre-defined
set of guidelines and a migration path, you
have something concrete to which you may
refer and measure progress against.
Poor planning
Planning is one of key factors that affect the
success of any project because “Fail to plan
is a plan to fail”. The project manager should
pay a lot of attention to this area and give it
enough time and effort regardless of time
pressure. They should be aware of bad results
when a project plan is non-existent, out of
date, incomplete or just poorly constructed.
To plan for a project is to set the
foundation for project work by defining
the tasks to be accomplished, and the time,
resources, staffing, communication and costs
involved in completing these tasks.
The quality of a detailed plan of work
depends on the project manager’s technical
expertise. Lack of such expertise will lead to a
much more generalised plan, in this case the
project should have a technical leader whose
responsibility it is to cooperate with project
manager to make detailed plans.
A successful project needs:
Risk plans
Every IT project involves some degree of risk.
Not doing an explicit risk assessment is one
of the major problems with project planning.
Projects that do not have a plan for
handling risks can be hit by sudden
unexpected events and be faced with
unachievable schedules and deliverables.
They can end up losing the client, and
because of that we realise the importance of
an adequate risk plan.
Risk management has become a major
issue especially as projects get bigger.
Success here means creating a plan to assess
the risks, the ‘which’, the ‘what’ and the ‘why’
of each risk identified and planned for.
Quality assurance plans
Projects must develop a QA plan as part
of the overall project plan to explain the
planning, implementation and assessment
procedures they will put in place to ensure
that project outputs comply with business
standards and best practice, as well as
any specific quality assurance and quality
control activities. A QA plan integrates all the
technical and quality aspects of the project
in order to provide a “blueprint” for obtaining
the type and quality of environmental
data and information needed for a specific
decision or use.
The project’s QA plan should cover the
issues listed in Figure 1.
When deliverables are supplied, the
project should also provide documentation
describing the QA tests performed and
evidence of compliance.
The more detailed planning the higher
the chances of success. Each and every
activity that is expected down the line gets
due attention.
Not only is this pre-planning well
documented, but also even after the project
has taken off, if things don’t exactly pan out
as planned, the project manager should
not hesitate to re-plan, avoiding project
management failure, and readily incorporate
the changed circumstances in their new
version, so that future events are controlled.
Figure 1: Issues in a QA Plan
Fitness for purpose
Deliverables should be fit for purpose. For example, projects should be internally
consistent, up to standard, free of bugs and perform well. This does not necessarily mean
perfection, but fit for purpose consistent with the level of funding and project resources.
Best Practice for processes
Projects should follow best practice for creating their deliverables, e.g.
technical design and architecture, programming, web sites, and data capture.
This should include processes, workflow, tools, equipment and methods.
Adherence to specifications
Projects will be asked to develop their own specifications. This might involve
requirements specifications, functional specifications, and/or technical
specifications. Once specifications are agreed, deliverables must conform to them.
Adherence to standards
Projects must ensure that their deliverables conform to company standards
for content, metadata, interoperability, terminology, learning and linking.
Accessibility legislation
Business systems should be accessible to a diverse range of users. In order
to achieve it we advise that all resources meet good practice standards
and guidelines pertaining to the media in which they are produced.
Project plans should consider cost,
resources and requirements needed to
succeed. These plans should be timed so that
there can be a monthly plan, a weekly plan,
and a daily task schedule so that everyone
can follow the progress of the project step
by step.
Poorly defined project scope
– unclear goals and objectives
A project manager should understand the
compromise between what they want to
accomplish and what they are actually able to
deliver. When goals exceed the ability to deliver
timely results, the project will surely fail.
Successful projects always have a welldefined scope that states realistic goals,
and attainable objectives, establishes clear
milestones, defines benefits and deliverables,
and conducts regular technical reviews
and measurements. By this you can ensure
that the project will be visible to all parties
including senior management and clients.
The scope should be clearly defined as
part of the project definition. Much of the
work at that time is directed at agreeing the
optimum definition of the project – both in
terms of its deliverables and in terms of how
it will operate. This scope definition will form
the baseline against which potential changes
are assessed and against which the project’s
performance is measured.
The concept of well-defined scope is
affected by many factors. For example the
goal of the project may be partially clear
because of poor requirements gathering
in the definition stage of project, goals and
objectives might be unclear because project
users lack the experience to describe what
they really require.
19
Project problems start with the three most
common scope mistakes:
¬
Overrunning initial cost estimations.
¬
Over – or underestimating project
schedule This is a double-edged sword:
setting a generous timeframe runs the
risk of the project becoming obsolete
by the time it is completed, but setting
a tight timeframe in relation to the
amount of work required will put a strain
on personnel.
¬
Miscalculating the work to
personnel ratio.
Vague requirements, poor user
input, lack of user involvement
Lack of user involvement will cause
a great deal of resentment among the
corporate user community, projects may
be seen as something forced upon them by
developers who only want to test out their
new toys. It should not be forgotten that
projects are built to support end users,
not developers.
Requirements need to be worked out
on both sides because there is a symbiotic
relationship between users and developers:
¬
Users, who know the business processes
best, need to clearly express their
requirements and provide feedback on
each project deliverable.
¬
Developers, who know what technology
can be used to put those business
processes into place, need to ask the right
questions and not make any assumptions
about what they think the users mean.
Nothing kills projects faster than giving users
something they did not ask for and then
pretending they did. IT teams may be given a
vague and informal set of requirements, and
they, in turn, may not bother to consult with
users or ask any questions, as a result they
will build what they believe is needed, not
what users need.
Scope creep, objective
and requirements changes
during Project
IT projects suffer from two classic problems
in project management, scope creep and
feature creep.
Scope creep refers to uncontrolled and
unexpected changes in user expectations
and requirements as a project progress,
while feature creep refers to uncontrolled
addition of features to a system based on the
incorrect assumption that one small feature
will add nothing to cost or time.
The project manager should understand
project trade-offs and make the right
decisions related to resources, features and
time schedule even though the requirement
changes. He should be aware of the risks of
change and the risks of not changing and
should have the ability to balance these risks
before deciding what to do.
One obvious solution is to establish a
reasonably stable requirements baseline
before any other work goes forward. But
even when this is done, requirements may
still continue to creep. No one can design
Figure 2: Example scope and change control process
Participants
Project
office
Identify
Capture
Review,
assess
Assign for
review
Propose
action
External
suppliers
Steering
committee
Contract revision
Approve
for action
Assign for
action
Action
Action
Review
action
Agree
closure
20
a process that assumes requirements are
stable. In virtually all projects, there will
be some degree of learning what the
requirements really are while building
the project. Projects could be headed for
trouble if architectures and processes are
not change-friendly, or if there are poorly
established guidelines that determine how
and when requirements can be added,
removed and implemented and who will
bear the cost of the changes.
On the other hand, if you build a project
from small, iterative phases instead of
mammoth, serial deliverables, you will
deliver more quickly, leaving less chance for
change to overcome the work, and less risk
of large project failure.
Another recommended solution for
scope creep is a change control process.
Change control will involve a combination of
procedures, responsibilities and systems. The
key to success is to have a well-controlled
but efficient process. Define and agree:
¬
On what basis changes should
be approved,
¬
Who does what,
¬
The membership of the change
control board(s),
¬
The detailed procedures, forms etc,
¬
Protocols for levels of authority, e.g.
what types of change can be approved
without reference to the project’s
business owners,
¬
Linkage to other management
procedures, e.g. the issue management
process, configuration management,
¬
Which tools will be used to support and
manage the process,
¬
How to communicate and promote
the process and its importance to
all participants.
Any participant or other concerned party
may raise Change Requests. The Project
Office team and Project Manager will ensure
they are captured and actively manage them
to conclusion.
An initial review should be made to
examine the need for change, how it could
be achieved and what the consequences
would be. The most appropriate member of
the Project Team would normally perform
this review. Based on those conclusions, the
recommended action would be proposed.
In this example, there are three possible
courses for the approval of the change:
¬
Minor changes within scope can be
approved by the Project Manager,
¬
Any change affecting an external subcontractor would need to be reviewed
with that contractor who would agree
any necessary contract revisions or
payments etc,
¬
Changes of scope and contract revisions
would require the approval of the
Steering Committee or the Change
Control Board.
In making the decision, the Project
Manager, Change Control Board or
Steering Committee would be guided by
pre-established principles for making
change decisions.
After the action is agreed the work is
assigned for action by the Project Team
and/or the external sub-contractor. When
complete, the action would be reviewed and
the Change Request closed. It is possible
that the agreed action could have more
than one stage. For example, it might be
better to introduce a temporary solution so
that the overall benefit from the project can
be delivered, and then build a permanent
solution after the system is live.
Poor architecture – inflexible and
difficult to change
Any environment usually develops, and
according to this development many issues
may change such as strategies aligned to this
environment objective, requirement etc.
The concept that “what we are using
today may be useless tomorrow” is clear
and understood. This concept should be
considered when building any project. If the
project architecture is inflexible for updates,
then this project may collapse because of
daily changes and rapid developments.
An example of flexible architecture is the
Patriot missile used during the Gulf War. It
was not designed to intercept scud missiles,
but the software was able to be reconfigured
to support the new function. On the other
end of the flexibility spectrum was a security
program created to protect sensitive wordprocessing documents. Everything worked
well for a few months until the operating
system was updated. The word-processing
programs still worked, but the security
program became useless and unfixable
because much of its code was tied to
operating system features that were dropped
in the new system.
People must think ahead about what is
likely to change. If you do architecture right,
you will not have to restart from zero again
and rebuild the project from the beginning
as nothing is existed because you are able
to add and modify features that caused by
any change any time, but if you do it wrong,
you will suffer death by a thousand cuts. Bad
choices show up as long-term limitations,
aggravation and costs.
21
Stakeholder conflicts
All the stakeholders of the project should
share similar business interests. For example,
assume that a project is being built, but
after a while the developers need some
clarifications, i.e., with input A, does the
system choose X, Y, or Z? If stakeholders
cannot agree on answers this will force them
to acknowledge deep incompatibilities
among their business interests, then the
system will be cancelled in an expensive
failure for the entire enterprise.
It becomes a problem when the
stakeholders work under the illusion that
everyone is going to get everything that
they want. They will contradict each other by
their differences rather than going through
conflict resolution in the early stages. The
developers will expose the stakeholders’
irreconcilable differences because
programmers cannot create an
ambiguous system.
Stakeholder conflicts can play many
different roles in project failures. Often,
stakeholders have personal reasons for not
being able to work together. When ego and
pride get in the way of any project, it will
almost always end in some disaster.
Other projects, especially smaller projects
within larger projects, never go anywhere
because the internal stakeholders cannot
agree on priorities. These are “pretend
projects,” meaning a few developers work
on them part time, but nothing is ever
delivered. Whatever the case is, you should
always think like this if you start any fixed-fee
project you should end it according to a
specific deadline, because it is important to
allocate budget ahead of time.
Lack of top management support
and involvement
Insufficient budget and poor
resource allocation
Few projects have the chance of getting off
the ground without the support of senior
managers in the organisation. Without
executive support the project managers in
the organisation will find it difficult to align
business requirement with their projects.
It is a problem when developers do not
know who the “real” sponsors are, and keep
progressing without sponsor involvement.
For the best true sponsors need to be shown
up and communicate with the team, follow
the project step by step, hear good and bad
news in “small pieces” rather than in “one
chunk”, this way you will avoid losing their
support if any surprise comes on the way.
Non-sponsored projects are taken less
seriously and may sometimes be viewed
as merely someone’s pet project. Without
the backing of senior management to lend
credibility to these projects, originators will
have a difficult time recruiting employees
to participate in development and testing.
Teams are usually made up of people from
different departments who all have their
own set of priorities and of course, they
all have their own bosses, so it is natural
that those involved in any project will have
tendency to keep the best interests of
their own department in mind, and there’s
nothing wrong with that. In fact, that’s why
they are on the team, to represent the needs
of their department. However the risk is in
having a selfish person or group who may
control project, ignoring requirements
of others.
Financial threats are the result of poor
budget forecasting and tracking, lack
of inter-department charge backs, and
ineffective tracking of resource and
cost allocations.
Insufficient budget is still a major reason
for missing goals and objectives of projects
within the quality framework that is required.
Project Y always needs to be delivered
tomorrow within X budget.
When we talk about budget we should
be aware of what may happen if there is not
enough funding, so a resource assessment
should be made carefully by conducting
complete and accurate financial analysis.
A resource assessment describes the
people, skills, hardware, software, and
network resources needed to complete a
project. Resource assessment is sometimes
the practical first step to making staffing
decisions for a project.
The project manager is typically
responsible for assessing resource needs and
deciding whether a formal, documented
assessment is necessary.
What kinds of projects need a
Resource Assessment?
Although every project undergoes some
kind of resource assessment, they are
frequently informal and undocumented.
Large, complex projects, and those working
with new technology, will benefit most from
formal assessment of resource needs. A
resource assessment needs to consider and
document the items in Figure 3.
Poor schedule estimation,
unrealistic or long timescales
Figure 3: Resource assessment contents
Project Name
22
Staffing & Skills
Inventory
What staff are already assigned to the project?
What skills do they have?
Roles/Skills Needs
What roles and skills are needed that
aren’t covered by project staffing?
Staffing Needs
What is needed to address roles and/or skills not
covered by staff already assigned to the project?
Training Needs
What training is needed to cover skill gaps?
Hardware &
Network Needs
What hardware and network resources
does the project require?
Software Needs
Does the project require any specialised software?
Support Needs
What kinds of support are needed from other C&C units
to address needs for skills and/or roles not covered by
project staffing?
Scheduling project work is an essential
element of project management. A project
schedule makes it clear to all participants
when work is expected to be completed. It
also shows the time-related dependencies
between different project tasks.
In a complex project, several schedules
may be necessary, covering different levels of
detail or different parts of the project.
Poor time estimation can cause project
related problems. One common problem
during the creation of the Work Breakdown
Structure is assuming that the time on task
equals duration. The time on task is the
time the task will take to complete without
interruptions, whereas duration is the time
the task actually take to complete including
interruptions. Using the time on task to
estimate schedule is a common mistake
made by project managers.
Who schedules the project?
Another common problem is using linear
approximation when estimating schedule.
For example, if you doubled the cows in a
farm, you double your production of milk.
The IT projects are beyond the scope of
such approximations. Assume we have a
large IT project using a team with a staff of
one hundred people. Linear thinking would
support the conclusion that increasing the
people by 100 percent would decrease
the schedule and increase the cost to
approximately the same degree.
In reality, doubling the staff produces
a non-linear result.
In general, every project has a minimum
achievable schedule. Many managers are
well aware of the need for fast delivery,
leading to other problems of unrealistic
timescales. These are set without considering
the volume of work that needs to be done to
ensure delivery. As a result these projects are
either delivered late or only have a fraction
of the facilities that were asked for or they
are bug-filled, because of that every project
manager should consider volume of work,
number of staff, number of working hours,
and the duration of each task in parallel to
avoid any kind of pressure. It is true that
working under pressure can increase the
quantity of results one receives, but, after
a point, dramatically reduces the quality of
those results. In fact pressure sometimes
produces the opposite of its intended effect.
On the other hand if the project
manager sets long time scales, the project
may be obsolete as a result of changes
in requirements.
Normally requirements change from
time to time due to changes within the
project users’ environment. If the project
objective is to serve certain society; it should
be parallel to their requirements. The key
recommendation is that the project time
scales should be short, which means that
larger projects should be split into
separate projects.
Setting overall completion dates must
be done by the project sponsor and
stakeholders. The project manager assists
by digesting information about scope,
deliverables, and resources, and estimating
times for completion of project tasks.
Once an overall schedule is set,
the project manager is responsible for
monitoring the progress of the project and
revising the schedule if needed. This must
be done in consultation with project team
members who are doing the work. Working
with team members to produce accurate
time estimates is one of the high mysteries
of the art of project management. The
project manager must balance the needs
for honesty and realism with appropriate
motivation to keep the project on track
despite inevitable surprises.
There will typically be give and take as a
project proceeds among budget, features,
and schedule. It is essential for the project
manager to keep all participants informed as
to current schedule status.
Time schedules should be reviewed
to see if they are realistic and participants
should be encouraged to express their
reservations on it.
Communication breakdowns
failure to communicate and
act as a team
Projects sometimes fail because of
inadequate communication between team
members; in such cases they lack the ability
to work as a cohesive unit and are in constant
disagreement. The arguments and infighting
cause everyone to move in opposite
directions, lowered morale, and spawn an “us
versus them” atmosphere.
Another common problem is the size of
the project team. There is a direct relationship
between the size of the project team and the
difficulty of keeping all members of that team
up to date on changes, progress, tools and
issues. Such problems are common on large
projects, especially if people are working at
different sites. In many troubled projects no
one person has an overview of the whole
project. Each project member needs to know
how his or her piece of work fits into the
entire architecture.
The key recommendation here is to
avoid forming a team of more than five
members, instead opting to form multiple
teams working on individual objectives. Each
of these smaller teams has a manager, who
is himself part of a management team. In
extreme cases multiple management teams
exist and an executive team is formed. The
focus of each team is rigorously defined and
strictly enforced/policed.
In general communications problems can
be avoided by adopting a communication
plan at the planning phase.
A communication plan identifies people
with an interest in the project (stakeholders),
communication needs, and methods of
communication. Communication planning
helps to ensure that everyone who needs
to be informed about project activities and
results gets the information they need.
The project manager is responsible
for identifying communication needs and
deciding whether a formal communication
plan is needed.
Although every project undergoes
some kind of communication planning, it
is frequently informal – determining who
needs to attend which meetings, receive
which reports, etc. Projects of long duration
will benefit from formal planning because
the project stakeholders are likely to change
over time. Projects that affect a large
number of people or organisations may
also benefit from formal planning to ensure
full identification of all stakeholders and of
communication needs.
A communication plan needs to consider
and document the items in Figure 4.
Figure 4: Communication Plan Contents
Project Name
List of Stakeholders
Who has interest in the project? See the project
definition for an initial list of stakeholders. Be sure to
include both business and technical stakeholders.
Information Needs
What kinds of information about the project are of
interest? Consider need to communicate plans, status
and progress reports, changes, major events,
availability of prototypes and demonstrations, etc.
Communication
Methods
What information will be communicated to what
groups in what ways? Common methods include
reporting and documentation, email, meetings, and
web sites.
23
Staffing – Insufficient number,
inappropriate skills
Staffing is one of the most critical elements
of a project’s success. Without staff, there
is no project. Once you have defined the
project and are clear about at least some
of the project’s initial tasks, you can define
your staffing needs. It is important to know
the type of staff that the project needs,
e.g. database administrator, one or more
programmers, and technical writer. Once the
type of staff has been defined, you need to
get individuals assigned to your project. The
best places to go for staffing resources are
the project’s sponsor and stakeholders.
You should be prepared to answer the
following questions that might come up
when you ask for staffing resources:
¬
What percentage of their time will
you need?
¬
How long will you need this person?
¬
What are the benefits of this particular
person working on the project?
¬
How do the skills needed and this
person’s skills match up?
¬
How many members do you need to
share workload?
Most IT projects require a diverse range of
skills; the project must have the right people
to do the right job.
For example, programmers need to
have experience in the technology before
counting on them, so they should be
selected wisely. Furthermore, managers can
perform poorly if they lead projects that
do not match their expertise. The project
manager should have enough experience
and knowledge from similar projects
before, so that the same mistakes will not
be repeated. Projects which deal with high
technology need managers with solid
technical skills. In such projects, authority
must reside with people who understand
the implications of specific technical risks.
However, the best technologists are not
necessarily always poised to be the best
managers. The skill set for management
and programming is disjoint. The larger
the project, the more need there is for
people with excellent planning, oversight,
organisation, and communications skills;
excellent technologists do not necessarily
have these abilities.
The solution to skill-driven challenges
is easy to define but difficult and expensive
to accomplish. A project needs to attract
and retain the most highly skilled and
productive people. A well paid project team
with the right specialised skills is worth
far more to an organisation than a group
of lower-cost people who need weeks or
months of fumbling through a new process
or technology before they can start being
productive. In a straightforward phrase “you
get what you pay for”.
24
Poor testing
The developers will do a great deal of testing
during development but eventually users
must run acceptance tests to see if the
project meets their business requirements.
This stage should be before the project
implementation, skipping the testing phase
because the project is way behind schedule
will lead to a downright failure.
However testing often fails to catch many
faults before a project goes live because:
¬
Poor requirements which cannot
be tested,
¬
Poorly or unplanned tests meaning that
the project is not methodically checked,
¬
Inadequately trained users who do not
know the purpose of testing,
¬
Inadequate time to perform tests as the
project is late.
Users should do the acceptance testing, in
order to build their confidence with a project
and to utilise their experience of the
business. To do so they need good testable
requirements, well designed and planned
tests, be adequately trained, and
have sufficient time to achieve the
testing objectives.
IT illiteracy
Sometimes adopting new technology may
lead to a failure, even though it is successfully
tested, implementing it for the first time in
the project is in itself a risk. Will the team use
it in the right way? Will they have enough
practice while they don’t have expertise? Will
it satisfy the project requirements?
It is related to the failure to align business
objectives with IT and its processes. This
usually occurs when the company’s internal
controls have material weaknesses or when it
is in non-compliance with various processes.
Therefore each project should have Internal
or external auditors who have an obligation
to publicly report facts.
Hidden costs of going
“lean and mean”
Any failure will be viewed as a direct
result of underperformance, even though
underperformance is not often a significant
factor in the failure of most projects. Instead,
failed projects often have goals that were
inherently unattainable, poor staff, etc.
Late warning signals
The early project milestones involve
diagrams, designs, and other documents
that do not involve working code, these and
other project milestones then go by or less
on schedule, and testing may start more or
less on time, so that errors which discovered
days before the deadline of the project will
cause the project not to be completed even
close to its deadline. |
Rasha Abdel Rahman
References
Department of Information
Technology, Audit Bureau of Jordan
Rasha Abdel Rahmman graduated
from the Information Technology
Faculty of the University Of Jordan
with a B.Sc Computer Science. She has
worked for the Audit Bureau of Jordan
since 2004 in a variety of technical and
managerial posts. She has specialist
skills as a developer and administrator
of Oracle databases. She is a qualified
IT Auditor and has represented the
Audit Bureau at international events.
Glaser, J (2004) Management’s role in
IT project failures Healthcare Financial
Management, October.
Grossman, Ira (2003) Why so many IT
projects fail, and how to find success
Financial Executive, Volume 19, Issue 3,
page 28.
Humphrey, W (2005) Why Big Software
Projects Fail: The 12 Key Questions The
Journal of Defense Software Engineering,
March Issue.
Armour, P (2005) Project Portfolios:
Organisational Management of Risk
Communications of the ACM, Volume 48,
Issue 3, page 17.
James P. Lewis Fundamentals of Project
Management, 3rd edition.
James P. Lewis Team-Based Project
Management in Back Matter (1), Back
Matter (2), and Back Flap
Betts, M (2003) Why IT Projects Fail [Online
journal] Computerworld, Volume 37, Issue
34, Page 44. Available from Academic Search
Premier at http://www.ebscohost.com
[Accessed July 21, 2005].
Jenster, P and Hussey, D (2005) Create
a common culture between IT and
business people to reduce project
failures Computer Weekly, March 22.
Coley consulting (2001-2005), Why projects
fail, Available at http://www.coleyconsulting.
co.uk/sitemap.htm
Simon Wallace (2004) The ePMbook,
Available at www.epmbook.com
Ephraim Schwartz (2004) online research
IT Myth 5: Most IT projects fail, August 13.
Paul Chin (2003) online research Cold Case
File: Why Projects Fail, May 6.
25
ad
Bornholm
(DENMARK)
Puttgarden
Kiel Bay
Kie
lC
an
al
Mecklenburger
Bucht
Kiel
Pomeranian
Bay
Stralsund
Rostock
Lubeck
Sea
Bremerhaven
Wilmhelmshaven
Swinoujscie
Hamburg
Schwerin
Emden
Szczecin
en
Elb
Ha
ve
l
e
Bremen
Oldenburg
Wittenberge
DS
Od
GERMANY
Ems
Braunschweig
nschweig
es
W
Frankfurt
e
Neiss
e
Essen
Halle
Dusseldorf
Cottbus
Elb
Dortmund
Leipzig
Gorlitz
Kassel
ogne
Dresden
Erfurt
EEisenach
Bad
Hersfeld
Siegen
Zielona
Gora
Spree
Dessau
D
Gottingen
ngen
rg
POLA
Magdeburg
Magdebu
er
Bielefeld
Munster
Potsdam
l
na
Hannover
Osnabruck
hede
Gorzow
Wielkop
er
Berlin
tella n dk
Mit
a
Werra
Jena
Gera
Bonn
Fuld
Chemnitz
Zwickau
Decin
Usti nad Labem
a
Koblenz
e
ell
Mai
Bamberg
Wurzburg
Rhein-MainDonau-Kanal
Mannheim
Rhin
e
Karlsruhe
Heilbronn
Plzen
CZECH
REPUBLIC
ava
Vit
Heidelberg
Saarbrucken
Prague
Cheb
n
Mainz
os
M
Hra
Kral
Hof
Frankfurt
Am Main
Wiesbaden
Nurnberg
Regensberg
Ceske
Budejovice
GERMANY
Dr. Ulrich Ditzen, Member
and Audit Director
in charge of IT Audit
highlights the general
structural IT problems
of German federal
departments and agencies.
He reports on the current
situation and procedures
in place, highlights the
shortage of staff with
adequate IT skills in the
public service, discusses
the importance of the use
of external consultants
for planning and
implementing IT projects
in the German federal
administration, the major
shortcomings made in
purchasing consultancy
services and the conditions
in which the use of external
consultants can add value.
26
Key issues
for relying
on External
Consultants
for Public Sector
IT Projects
Audit findings generated by the
German SAI on general structural
IT problems of German federal
departments and agencies
include computer literacy, experience with
suppliers and with relevant legal practices.
The problems facing us can be illustrated
by a quote from the budget documents of a
German federal government department:
90% of the IT budget has been definitely
allocated to operation and maintenance.
IT-related staffing is often quantitatively and
qualitatively inadequate to meet current
requirements. Either the number of posts
is insufficient or posts are vacant and it is
difficult to recruit adequately skilled staff.
The use of external staff may be a
potential solution.
There are a number of recurring
problems and factors critical to the success
of IT projects. The objectives of projects
are often not clearly defined in terms of
content, use of resources, value added and
specific requirements. It is often difficult to
identify the biggest common denominator.
Deadlines are often set first without stating
how to accomplish the objective. Staff
with operational responsibilities are often
not asked or do not clearly state their
preferences. No investment appraisal is
carried out.
IT project implementation periods
are often so long that, given ever shorter
innovation cycles, there is a risk that
specifications laid down initially become
outdated and obsolete while project
implementation is still underway.
Management consulting firms often point
out the following recurring problems and
factors critical to the success of IT projects:
skills available in-house are often limited, so
is expertise in procurement and contractawarding procedures, in project steering and
the later transfer of know-how. Changing
specifications and managing the changes are
often impaired by the shortage of adequate
skills among public sector staff. These skills
“In recent years, the degree of
IT integration in operational
functions, the intensity of IT
support, the connectivity between
workplaces at branch offices
and headquarters, the degree of
information density but also the
dependence of stable operations
on the high quality of IT systems
have steadily increased.”
Further factors to be considered are:
¬
Between 1995 and 2003, total IT
expenditure (budget title groups 55 and
56) increased by more than 150%. Broken
down by capital expenditure on IT and
recurring expenditure on IT support
services, these increases are 175% and
450% respectively.
¬
There are 15 federal government
departments (ministries) with a total of
435 agencies. There are 211,000 federal
workers (137,140 civil servants and 73,875
employees, excluding the Armed Forces
and Federal Employment Agency).
Total budget expenditure in FY 2007
was about €260 billion. As far as can be
ascertained under the present budget
system, specific IT expenditure totals
€2 billion. However, we have reason to
believe that an undetermined amount of
IT-related expenditure is hidden under
budget items that belong to construction
projects and to the performance of
operational functions by departments
and agencies.
¬
As of early 2007, the civilian departments
and agencies were equipped with
140,000 personal computers, while
135,000 personal computers were in
place within the remit of the Ministry
of Defence and 95,000 in the Federal
Employment Agency. In the federal
ministries, nearly 100% workplaces are
equipped with personal computers.
In the subordinate agencies, more
than 90% of workplaces are equipped
with personal computers. Since 1999,
departments are fully connected by a
single intranet, which was built up in
connection with the relocation of the
seat of the Federal Government and
Parliament from Bonn to Berlin.
Audit findings generated by the
German SAI on general structural
IT problems of German federal
departments and agencies
IT services have become vital for government
operations and transactions. However, IT
applications and structures have for many
years evolved as isolated solutions with an
exclusive focus on the operational needs
of the department or agency in question.
The ’patchwork’ grown over time had to be
harmonised, as compatibility problems of the
applications and a large number of (unused)
IT system functions resulted in a lack of
acceptance and in ‘frictional loss’. Systems
integration has become ever more important
for suppliers and service providers but above
all for the purchasers.
Audit results generated by the
German SAI on problems with the
availability of staff
Frequently, the project staff do not have
adequate skills and the project leader does
not have any solid experience. Staff are not
released from their previous functions or are
assigned to a project for different periods.
Successful support by a coach would
require intensive participation, steering
and acceptance of the services delivered. A
clear strategy as to developing skills needed
in-house has often been lacking. The transfer
of consultants’ expertise has often not been
stipulated in an extra contract.
The importance of external
consultancy for public-sector
IT projects
IT projects have become increasingly
complex and their implementation requires
more input. This is due to higher quality
requirements, the widening scope of
functions to be performed, the increasing
number of stakeholders, the need to take
into account existing IT systems and the
stronger integration between different IT
systems. Project objectives often conflict in
terms of performance, time and resources.
Until recently, no generally accepted
definitions of the terms “experts” and
“support services” existed within the German
federal administration. Other terms such as
“assessor”, “business advisor”, “consultant”
and “coach” have frequently been used
indiscriminately without much regard for a
precise and uniform definition.
In 2005, the German SAI developed,
in agreement with the Federal Ministry
of Finance, a definition of “external
consultancy”. According to this definition,
the subject of external consultancy is the
provision of a service against remuneration
with the objective of developing, assessing
and imparting to the decision-makers
practical recommendations with respect
to concrete decisions to be taken by
the contracting authorities and, where
appropriate, providing further advice
during implementation.
In the context of this definition, recipients
of the consultancy services are federal
departments and agencies, quangos and
grant recipients. The service provider is a
natural or legal person active outside
this field.
The German SAI’s audit work has revealed
that consultants are primarily relied on in the
following stages of IT projects:
1. identification of requirements,
2. drawing up of specifications,
3. valuation / estimates of costs and
expenditure,
4. negotiations with contractors,
5. change request procedure,
6. review and revision, risk assessment,
7. testing and acceptance.
The administration often justifies its reliance
on consultancy services with the following
arguments: Funding can be obtained more
quickly because funds for “procurement”
are easier to obtain than funds for hiring
staff. A consultancy contract is usually made
for a limited period of time. This improves
the chances for a project to be approved,
facilitates quick implementation and the
overcoming of internal conflicts. The use
of external consultants also enhances the
legitimacy and prestige of projects, often
generates new ideas and facilitates the
discovery of other options. On rare
occasions, external consultants serve as
trouble shooters.
27
contract award procedure, project results
Audit findings generated by the
implementation and use
German SAI on general IT-related evaluation,
of results.
staffing problems in German
federal departments and agencies Audit experience in connection
with the implementation of
As a result of the increasing use of external
staff, private-sector staff are used for long
results generated by consultancy
periods of time to perform inherently
administrative functions. There is a trend
to contract out even sophisticated and
conceptual IT functions to the private
sector. External (private-sector) staff
permanently perform functions of everincreasing importance.
Where external staff are used to evade
internal staffing problems, this creates
a high and increasing dependence on
external expertise in an environment of
rapid technological change. Departments
and agencies increasingly lose the ability
to assess and act upon emerging issues.
Declining budgets, increased cost pressure
and shortage of staff resources increasingly
restrict the scope for government action.
Shortcomings found by the
German SAI concerning the use
of consultants
There is a general trend to rely on
consultants also for the performance of core
functions. The contracting authority often
has inadequate competence for controlling.
Mistakes are made most frequently during
the following stages and in the following
fields: planning of the use of consultants
(identification of the need for consultancy
services), performance (value for money),
28
In a number of cases, the German SAI has
had doubts as to whether results generated
by consultancy had the intended effect or
were suitable as a sound basis for decisions.
These doubts were based on the criterion
that successful necessary consultancy should
have a clear influence on further action,
current and future decision-making.
Scientific study
A scientific study carried out in the public
sector revealed that the feasibility of the
problem solution worked out is the most
important criterion for selection (80%).
However, the criterion is ultimately met in
only 50% of the cases reviewed. The study
further found that the proposed solution
was often implemented only “to a small
extent” and that, in nearly one third of the
cases, consultants had to remain active
during implementation. The know-how
expected to be generated was transferred in
only two thirds of the cases reviewed.
Summary and outlook
(Core) functions that should be reserved to
the public sector include developing and
deciding on the IT strategy, IT portfolio, IT
architecture, IT standards, IT controlling and
reporting, IT procurement and management
of IT interfaces.
In its audit work, the German SAI has
found that the factors critical to the success
of consultancy for IT projects and the
measures necessary in this context are:
1. the definition of problems and objectives,
2. the necessity of consultancy,
3. performance (value for money),
4. the specification of requirements,
5. the methods of awarding contracts for
consultancy services,
6. the precise formulation of contractual
provisions,
7. monitoring, steering and acceptance,
8. the implementation of the results
developed by consultancy.
To address an IT problem, the administration
should accurately analyse and determine
the current situation, the objective to be
accomplished and the difficulties emerging
or identified in achieving the objective.
The German SAI’s audit findings highlight
the fact that, on the whole, the decisions
about the use of external consultants are too
rarely based on a sound and sensible analysis
of the problem and that the objectives and
criteria have often not been determined in a
way permitting project evaluation.
Before considering the award of a contract
to an external service provider, the
administration should critically examine
whether it can perform the service itself.
In the course of its audit work, the
German SAI has found that, in many cases,
departments and agencies give reasons for
the use of consultants that are not directly
connected with the problem to be solved,
e.g. “opening up other perspectives“ or
“better way of convincing the policymaking level of the validity of results
[generated by third parties] on which
action should be taken“. Such reasons often
were more important than the need to
compensate for a lack of professional or
technical in-house expertise.
It is always necessary to carry out an
investment appraisal, in the course of which
all alternative options need to be stated
and evaluated. The Bundesrechnungshof
found that, preliminary to awarding
contracts to external consultants, investment
appraisals, which are a necessary tool for
verifying compliance with the requirements
of efficiency and effectiveness, have
rarely been carried out. Based on the
German SAI’s experience, there are the
following alternatives to commissioning
external private-sector consultants: Apart
from performance of the service by the
department or agency itself, support
can be obtained from, for example,
internal consultancy teams of the public
administration from their own or another
government department.
If purchasing external consultancy
is the most cost-effective option, the
administration needs to unambiguously and
comprehensively describe the consulting
service to be performed. If the administration
itself is not capable of describing the service
required, this is evidence of the fact that the
project is not yet ‘mature’ enough for calling
in external expertise. Where the department
or agency in question is not able to describe
the problem to be solved or – as frequently
observed – needs a third party to describe
the problem, it will also not be able to verify
whether a specification of requirements
drawn up by an external consultant actually
meets the requirements of the department
or agency.
As a matter of principle, a public invitation to
bid must be issued and, where appropriate,
such invitation has to be published
throughout the European Union. A contract
award by negotiated procedure is admissible
in few exceptional cases only. However, in
most of the cases audited by the German
SAI, contracts for consultancy services were
awarded without competition.
Contracts must be worded so as to
ensure that the content and timing of
the service purchased are described
unambiguously in a way permitting
verification. Prior to awarding a contract for
consultancy services, there should be full
understanding of the essential contents of
the contract. This includes the description of
both the service to be provided and of the
results aimed at. Sound evaluation criteria
and milestones defined in detail should be
available at an early stage.
By appropriate monitoring and
steering of the consultancy activities, the
administration can make a substantial
contribution to their success.
The results generated by consultancy
have to be accepted on a timely basis in
order to safeguard the possibility of
claiming damages for breach of contract
or poor performance.
The Bundesrechnungshof has found
that departments and agencies adequately
monitor and steer the consultancy projects
only in a few cases, that the acceptance of
the service was frequently delayed, that
ambiguous specifications such as “assisting
the contracting authority with … “ or “…
will be available as consultant beyond that
period“ hamper any effective monitoring
of service provision. Such formulas are no
appropriate basis for accepting the service.
A reliable and transparent ex post
project results evaluation should be carried
out after the conclusion of any consultancy
project. In many cases, the results
evaluations required under budgetary law
had not been carried out.
The German SAI often had doubts as to
whether results generated by consultancy
had the desired effects or whether they were
used at all as a basis for decision-making.
On balance, budget funds were spent
on consultancy work whose results added
little or no value; due to omitting ex post
results evaluations, no lessons were learnt to
prevent deficiencies in future similar cases.
The administration needs to
transparently document each successive step
from the description of the problem via the
verification of needs to the implementation
of the results generated by consultancy. This
is an indispensable prerequisite for carrying
out any ex post project results evaluation.
In the course of its audits, the German
SAI frequently found inadequate and
incomplete records.
Comprehensive documentation avoids
duplication and provides key information for
the staff assigned to an IT project later on
and for the planning of future projects.
The use of consultants may effectively
support administrative activities in cases
where problems cannot be solved in-house.
The extent to which external consultants
are used will continue to increase,
especially in the case of major IT projects.
Nevertheless, the risks and potential errors
are as manifold as the possibilities for
making use of external expertise.
The decisive factor is the ability of the
contracting authority to identify its own
needs and to monitor the provision and
success of the service purchased.
Consultancy services are not a
commodity whose choice is merely
governed by its price (as in the case of
hardware or IT infrastructure services).
It is absolutely necessary for the decisionmakers to be aware that the principles of
economy and efficiency also apply to the
use of external expertise (need, efficiency,
competition, evaluation of results).
Alternative options would be service
centres within the administration having
the necessary special skills and expertise
needed to cover the entire remit of a
department or even to perform crossboundary advisory functions. |
Ulrich Ditzen
Ulrich Ditzen is a graduate of
Darmstadt Technical University where
he earned doctorate in economics in
1980. He is a Member of the German
SAI, the Bundesrechnungshof and is
currently Audit Director of the unit in
charge of IT auditing. He joined the
Bundesrechnungshof in 1987 and held
posts in various audit units such as
telecommunications and electronic
accounting. Prior to that, he worked at
the Federal Ministry of the Economy.
29
û
8
8 û
6
rojo
Kiruna
ki
Kolari
Kelloselka
Ou
nas
Kamilarvi
ea
joki
rn
To
Malmb
erg
et
odo
B
n
lve
Jokmokk
Lu
Ke
mijo
ki
Rovaniemi
Mo
leal
ve
Sandnes
San sjoen
n
Kemi
oden
B
Mosjoen
Sk
elle
ftal
ve
n
Norwegian Sea
Iijoki
Lulea
ite a
P
Oulu
Ou
lujo
ki
Storuman
Skellefte a
û
4
6
64û
Lycksel
Vilh
elmina
Um
ea
Nams
N os
analven
Steink
jejer
Steink
Kajaa
lve
n
Kokkol a
Umea
Angerm
SWEDEN
Storsjon
Trondh
ei m
FINLAND
Ornskoldsvi
Indalsa
lven
K
k
Vaasa
Ostersund
Molde
Alesund
e
Seinajoki
Aanekoski
Jyvaskyl
Andalsnes
Harnosan d
Ang
e
NORWAY
Sundsval l
Ljusnan
Ost
Glama
Floro
erd
ala
Hudiksval
lve
n
l
Gulf of
Bothnia
M
Jamsa
n
Lage
or i
P
Lilleh
ammer
Tamp
er e
Hameenlinna
Rauma
Lah
t
Mora
Klar
Hamar
erg
B
en
a
Gavle
Falun
n
alve
60û
Turku
Dal
orlang
B
e
alve
Helsi
n
Drammen
Up
sal
p
Oslo
Karlsta d
Otra
Stavang
er
Vanern
g
a
Sk
û
6
5
Vannersb
or g
Uddevalla
ak
err
Goteb
or g
Hiiumaa
Motala
Falkop
ing
oras
B
Jonkop
ing
f
f o
Fin
lan
d
ESTO
Nykop
ing
arnu
P
Norrkop
ing
Linkop
ing
Gotaalv
Gul
Tallin
Sodertalje
Mariesta d
Mellerud
Arendal
ALAND ISLANDS
Stockholm
Oreb
r o
orsg
P
runn
Kristiansand
a
Vasteras
Haug
esund
Saaremaa
Vattern
Vastervi k
Nassj o
V
il
North
Sea
Klaip
eda
LITHUANIA
û
4
2
Effective IT
Governance:
How to Get Good,
Secure IT Services
SWEDEN
This article describes
experiences from 19 audits
of the Government’s and the
public administration senior
managers’ IT governance. Our
main conclusions are that there
is an urgent need for stronger
IT governance at both the levels
of the Government and senior
managers. Only such governance
can ensure that good, secure
IT services will be conceived,
developed and implemented,
as well as meet all significant
requirements for IT security.
Since the audits were conducted
actions have been taken at both
levels to strengthen
IT governance.
In Sweden, according to the Government,
government agencies should become
proficient information technology (IT) users,
especially in two areas: (1) good e-services,
as part of e-government, and (2) security of
these services, that is, the protection of the
confidentiality, integrity, availability,
and traceability of data, as well as the
protection of IT systems. The Swedish
National Audit Office (SNAO) audited the
performance within these areas, performing
19 audits, from 2002 to 20071. The IT
Governance audits can be classified in
three audit areas:
1. Effective IT-Based Investment in
Business Change – Focus on Agency
senior managers
2. Effective Web Sites and Good
e-Services – Focus on the IT Governance
of the Government and Agency
senior managers
3. Effective Security for Information Assets
and especially for e-Services – Focus
on the IT security governance of the
Government and Agency
senior managers
Audit Question
Did the agencies manage investment
in IT-based business change so as to
achieve efficiency?
Methods Used
Our audits were based on the IT investment
management model (ITIM) of the U.S.
Government Accountability Office
(GAO), supplemented by Swedish legal
requirements and adapted to the Swedish
administrative environment. This adaptation
was key in making it easy for senior
managers at different levels in the agency
to understand the audits. During the audits,
we noted that senior managers did not have
any problem relating their work to our norm.
The norm includes agencies’ operational
activities, such as strategies, with the
requirements for each:
¬
Develop proposals: An innovation system,
built on activities that are well managed
and developed, which produces good
investment proposals, including those for
IT support2.
¬
Assess proposals: (1) Investment
proposals include proposed development
programmes (for example, for IT support)
and (2) assessments based on an
agency’s available IT resources
(including a database).
¬
Select proposals for implementation:
New proposals are related to earlier,
ongoing and approved development
programmes (so-called “investment
portfolios”), so as to guarantee links
to the (1) investment strategy and (2)
evidence trail for tracking decisions.
They are described below, together with
some lessons to be learned.
Audit area 1: Effective IT-Based
Investment in Business Change –
Focus on Agency senior managers
In this area, we audited the IT governance
of senior managers in five agencies heavily
dependent on IT: the National Labour Market
Administration, the National Land Survey,
the National Road Administration, Statistics
Sweden, and the Swedish Meteorological
and Hydrological Institute. In particular,
1
2
30
we looked at IT governance in terms of the
steps senior managers took to assure good
investment in IT business change.
Until June 30, 2003, there were two public audit offices in Sweden: Riksrevisionsverket (RRV) and Riksdagens
revisorer (the Parliamentary Auditors). On July 1, 2003, these two offices were amalgamated to form Riksrevisionen
(RiR). The RRV and the RiR have the same English name: Swedish National Audit Office (SNAO).
An innovation system consists of a network of groups, organizations, people, and rules in which new processes
and methods are created.
¬
Manage implementation: (1) Programmes
are given realistic conditions for
success, (2) project risks are assessed
and managed, (3) standards are used
consistently, and (3) completed projects
are monitored.
¬
Knowledge management: Good use
made of the experience acquired
to continuously improve the
investment process.
¬
Create and maintain the investment
process: Sufficient oversight of the
investment process, identifying
strengths, weaknesses, and
possibilities for improvement.
For each audit area we used these methods:
asked the senior managers to answer a
questionnaire with self-evaluation questions,
asked for relevant documents showing
the agencies’ activities for each strategy in
our audit norm, analysed the answers on
the self-evaluation questionnaire and the
norm-related documents, interviewed 15 to
25 staff, drafted an audit report and asked
for agency comments, gathered agency
representatives to a special seminar in which
both the identified problems and possible
solutions were discussed, and informed
senior managers of our findings and
recommendations.
Audit Findings
We found that the five agencies, despite
their long experience with IT investment,
had considerable shortcomings in the
governance of IT investment (see IntoIT
issue 18 Better managed investment in
IT-based business development). These
agencies lacked:
¬
sufficiently well-developed processes
to elicit good ideas as to how IT can be
effectively managed;
¬
periodic, systematic reviews of their
investment processes, enabling them to
identify where change is needed;
¬
adequate articulation of their investment
strategies, making it difficult to justify and
select among competing proposals;
¬
obtaining a clear and comprehensive
understanding of an investment
proposal;
¬
¬
business management driven projects
in combination with well-established
methods and models for managing and
undertaking investment project; and
achieve the anticipated benefits of IT
investments in an agency’s operations.
Shortcomings in investment strategies
created problems when translating the
assessment of IT investment proposals
into approved decisions. Because the
investment proposals did not link well with
the operational strategies, the risk increased
that the proposals would not lead to the
investments sought by each agency. In
addition, investment decisions were not
always based on clear descriptions of the
proposal’s expected business benefits
and implementation risks. Furthermore,
proposals setting out the comparative costs,
risks, and effects of alternative approaches to
IT investment projects were not adequately
dealt with, nor were proposals clearly linked
to each other. These combined factors
prevented decision-makers from obtaining a
clear and comprehensive understanding of
an investment proposal.
Moreover, IT projects were inadequately
integrated into (1) previously approved
investment projects and (2) the IT systems
– the environment – in which they were
intended to operate or which they were
intended to support. An IT investment
alone rarely achieves the anticipated
benefits in an agency’s operations. It is often
necessary to change working methods, staff
development and organisation. In addition,
governance of the IT projects was carried
out at too low a management level. This
meant that the governance of individual
business projects was more geared to
reacting to problems that arose (reactive
management) rather than to systematic risk
assessment (proactive management). With
systematic risk assessment, an environment
is created and maintained in which risks
are not allowed to develop into problems.
Finally, well-established methods and
models for managing and undertaking
investment projects, such as those identified
in the IT investment management model,
were not used consistently. Experience and
knowledge of different components of the
investment process were not utilised in a
systematic way, which all the agencies in
our audits acknowledged to be an area
for improvement. In addition, we found it
difficult to (1) obtain an overview of the
knowledge that exists and (2) gain access to
the knowledge when needed. In particular,
only one of the agencies had utilised lessons
from past investment projects for new ones.
Recommendations
In general, all five agencies should improve
each step in the IT investment process. In
addition, the Government should exert
better governance of government agencies
that are concerned with IT investment.
3
32
Audit area 2: Developing Effective
Web Sites and Good E-Services
– Focus on the IT Governance
of the Government and Agency
senior managers
In audit area 2, we audited the development
of e-services, asking detailed questions
concerning the development of effective
Web sites and good e-services. As part
of this audit area, in 2002-03, we initiated
audit project A. Two risks were defined in a
pre-study: (1) the digital divide and (2) poor
usability of Web sites and other services,
which were squeezed out by investment
in e-services. In 2003, we initiated audit
project B, a materiality and risk analysis of the
government’s IT governance of the transition
to e-government – that is, 24-hour, 7-day
government agencies. We found eight main
risk areas3:
¬
overall governance of government
agencies’ work on e-government;
¬
agencies’ implementation of
e-government;
¬
administration and operation of the
infrastructure for different types of
services;
¬
use of e-services;
¬
the effects of investments in
e-government;
¬
the support for the work on
e-government;
¬
the sources – what are they?
– and purpose of the current fashion of
investing in e-government; and
¬
technical advances as a foundation (that
is, the development of components for
Internet applications) for e-services.
Audit Questions
For project A: How effective are agency Web
sites in meeting the needs and requirements
of the individual user?
For project B: How effective are the
Government and government agencies in
developing good e-services?
Methods Used
For project A, we used several methods: a
Web questionnaire sent to 92 government
bodies, in-depth interviews with immigrants
and elderly people and a test of 92 Web sites
using national and international accessibility
standards and our own criteria for special
categories of users.
For project B, we investigated all
levels of the government: the demands,
requirements, e-policies and strategies
from the Parliament and the Government.
We performed interviews focusing on
the interaction between the Government
and agency senior managers concerning
the direction of the development of
e-government, and the agency Senior
We have not analysed risks from the Swedish Parliament’s point of view, for example, risks related to democracy.
Manager’s strategic analysis and actions
based on direction of Government. We did
10 case studies, divided among government
agencies and related government
departments. These case studies included
in-depth study of Web sites (for incoming
e-mail, information quality, and initiatives for
new e-services).
Audit Findings
For project A, we found that the agencies’
Web sites and the e-services offered there
did not promote an efficient dialogue
between users and agencies. In particular,
the Web sites failed to meet certain
accessibility requirements for the disabled,
immigrants, and the elderly.
For project B, we found that the
governance of the Government for investing
in good e-services, including the types of
e-services to which the agencies should
give priority, was limited. Instead, the
Government chose to exert governance
mainly through its own support agencies
and by means of rules, which were
inadequate. In addition, the Government’s
reports to the Swedish Parliament contained
no information about the effects of
e-government, including e-services.
We also found that government agencies
had difficulty in developing good e-services
because they lacked government support.
As a result, e-services have not been
developed; do not meet user requirements;
and are at risk of citizens’ mistrust, given that
the agencies, as well as the Government, can
not guarantee security, especially for e-mail
to the agencies. In addition, at the agencies,
narrow reasoning was allowed to govern
investment. Agencies had to finance such
investment entirely from their own resources.
This created poor incentives to build eservices in collaboration with other agencies.
Finally, certain legislation made it difficult
to achieve an effective use of e-mail and
Web sites. We found e-mail – a basic service
of e-government and the most important
route for citizens wishing to contact their
government – a particular problem. Citizens
demand to be able to use e-mails as a means
of formal communication, but agencies are
not legally bound to answer e-mail or attend
to e-mail enclosures.
Recommendations
The Government should improve
interagency collaboration, which
requires more elaborate governance of
communication among agencies. The
Government should also appreciably
improve its control of agency modernisation
efforts, including the establishment of
clearer rules and guidelines, so as to enable
e-government for government agencies’
handling of e-mail.
Audit area 3: Effective Security
for Information Assets and
especially for e-Services – Focus
on the IT Security Governance
of the Government and Agency
senior managers
In audit area 3, we audited IT governance
of e-services security. As part of this audit
area, in 2005-06, we initiated audit project C.
In particular, we looked at whether senior
managers systematically used internationally
accepted standards for information and IT
security. In 2007, we initiated audit project D,
an analysis of the Government’s governance
of the public administration’s field of actions
in the area of information and IT security.
In audit project C, we audited senior
managers’ governance of information
and IT security. The information and
IT security is concerned with:
¬
protecting information assets against
manipulation and destruction;
¬
preserving information assets availability;
¬
preserving information assets
confidentiality; and
¬
preserving an audit trail concerning
information assets used.
This security is especially important now
that e-government is opening up agencies
to threats from the outside world. For this
reason, we carried out audits in 2005 and
2006 of IT security at 10 major government
agencies with significant information assets.
In the audits, we focused on senior
managers and their governance of IT security.
This means that we studied senior
managers’ IT governance of security, including:
¬
control environment;
¬
risk analysis;
¬
control functions and individual security
measures;
¬
information and training; and
¬
follow-up, evaluation, and further
development and administration.
In audit project D, we audited the
Government’s governance of information
and IT security within the public
administration. The audit was carried out in
the light of the problems that have emerged
in the SNAO’s audits of ten public agencies’
performance of their responsibilities for
information security (audit area C).
Audit Questions
For project C: Considering the prevailing
standards for information security
management systems, is the government
agencies’ IT security governance effective?
Given the audit question, there
were two possible areas to be audited:
(1) actual security and (2) senior
managers’ IT governance of security.
We chose to focus our audits on senior
managers’ IT governance of security.
For project D: Is the Government taking
its responsibility for making requirements of
and following up the work of their agencies
(the public administration) with respect
to security of information and IT, and for
taking the initiative for measures aimed at
improving the conditions for the work of the
public administration within this area?
Methods Used
For project C, we used several audit
techniques: (1) a Web questionnaire to
get agencies’ opinions about their IT
security; (2) a request for formal documents
showing the agencies’ security activities
at all organisation levels (we received 50
to 100 different documents from each
agency); (3) follow-up concerning the
documents; (4) study of the questionnaire
answers and the documents; and (5) 10 to
15 interviews, focusing on senior managers
(interview questions were based on a
special questionnaire, related to the COSOstructure). Finally, we drafted an audit report,
letting each agency comment on the draft
and informing the senior managers about
our findings and recommendations.
We took as our starting point an
international standard (ISO 17799), and
added components from Swedish legislation,
as well as international experience. We then
transferred the requirements for IT security
to a COSO perspective which means that
we examined senior agency management’s
internal control and monitoring of
information assets and IT security.
For project D, we used several methods:
we analysed the findings from the 10 audit
projects in order to ask the Government if
the common pattern of problems among
the 10 audits was known or not, we gathered
information concerning our pattern of
problems from four agencies being expert
and used by Government in the area of
information and IT security, we analysed the
Government’s written statements in official
documents to the Parliament concerning
the status of information security and what
actions the Government had promised to
take, we performed in-depth interviews
(based on questionnaires) in the Government
focusing on the information gathering and
organising of matters concerning information
and IT security. We also made a special
analysis of shortcomings in the legislation
in the area.
33
Audit Findings
In project C, we found that government
agencies were not working effectively
because important parts of the information
security management systems were missing
or defective:
¬
Control environment—organisation of
security work, policies, and reporting
Senior managers’ attitudes (1) were
not always favourable towards security
investments, (2) did not show a keen
understanding of today’s threats, and
(3) did not always formulate clear
security objectives.
¬
Risk analysis: Often patchy, seldom
comprehensive. Following the
implementation of investments in
security measures, senior managers
often did not demand an overview
of important and residual risks.
Responsibility often unclear, and
methods for analysis not selected
and decided.
¬
Training for skills: Priority was given to
technical measures rather than training.
Education seldom systematic, including
that for staff who need refresher
knowledge about (1) their responsibilities
and (2) how, if there are problems,
troubleshooting should be carried out.
¬
Chain of command: Reporting upwards
was not well organised.
¬
Cost: No one senior manager had a clear
picture of the costs of IT security.
¬
Senior managers’ responsibilities:
Inadequate follow-up on the
implementation and operation of security
measures that had been decided earlier.
Finally, the information security management
systems are not comprehensive—that is,
responsibilities, reporting, and follow-up
are not integrated. Important objective
data, with which senior managers make
decisions, was missing. This made it hard
for senior managers to exert effective IT
governance of security. Therefore, the
potential for investment in IT security is not
well exploited. The amount of resources
invested and the costs were most often not
even known!
34
In project D, we stated that the
problems on agency level described
above were serious and that they imply a
risk of significant negative consequences
for government commitments such as
electronic government and national
emergency management. In the light of
the above, the SNAO considers that the
Government’s control of information security
is of great importance. The SNAO’s overall
assessment is that the Government has
not followed up to ensure that the internal
management and control of information
security in the public administration is
satisfactory. The Government has not
taken sufficient initiative to improve
the conditions for the administration’s
work on information security.
The SNAO has established that the
Government has taken measures with
respect to the technical conditions for
agencies’ information security work, such as
e-signatures, e-identification, secure Internet,
etc. On the other hand, no measures have
at the time of the audit been taken to
support the agencies’ internal management
and control of information security.
The SNAO takes the view that an overhaul
of the regulations is urgently needed,
particularly against the background of
the investment in e-government.
The Government has not given the expert
agencies a sufficiently explicit mandate,
which has meant that they have had
difficulties in giving the Government
a complete picture of the information
security problems at the agencies.
An explicit mandate is also needed in
order for the expert agencies to provide
appropriate regulations detailing the
Government’s requirements for the
agencies’ work on information security.
The audit shows that over the past ten
years the Government has been broadly
aware of certain management problems
in the sphere of information security, but
the picture has been unclear with respect
to central government agencies and the
Government has been unable to present
any complete picture of the problems
affecting the public administration.
According to the SNAO, the Government’s
organisation of the work done by the
Government Offices on information security
issues and the management of the expert
agencies is together insufficient to handle
the agencies’ problems with their
information security.
Recommendations
For project C: Senior managers’ control in the
field of IT security should be strengthened.
This could be done using the standard
SS-ISO/IEC 27001/17799 Information
Security Management. One key activity
is the risk analysis. This activity needs to
be strengthened since it is the base for
information security measures.
For project D: The Government should
focus more clearly on information security
issues. Give the expert agencies an explicit
mandate to follow up and report on the
agencies’ work on information security. Give
the agencies better conditions - set more
explicit requirements for information
security work.
Lessons Learned
As a result of the Government’s investment
in electronic government, growing numbers
of agency services are becoming available
on the Internet, agencies are joining
together to create co-ordinated e-services
and there is a general increase in IT-based
development work. In order for this reform of
the public administration to succeed, citizens
and businesses must have confidence in
the e-services provided on the Internet.
There is a risk of a lessening of confidence in
the agencies’ e-services if the information
cannot be protected. It may be a case of
unauthorised persons gaining access to
sensitive information or changing data or in
some other way acting so that the services
cannot be used. If that happens, there is a
considerable risk of the entire investment in
e-government being jeopardised.
In the transition to e-government, in our
opinion, there is an urgent need for stronger
IT governance at both the levels of the
Government and senior managers. Only such
governance can ensure that good, secure
IT services will be conceived, developed,
and implemented, as well as meet all
significant requirements for IT security.
Since the audit projects been finalised
in spring 2007 we have made some
follow-ups. At the agency level we noticed
some improvements of IT governance
of information security in form of plan of
actions, reviewing important documents,
implementing information security
standard and educating the staff. During
autumn 2007 an expert agency published
regulations stating that government
agencies should implement an information
security management system. At the
Government level a plan of action to improve
e-government recently (February 2008) has
been taken. In this plan of action the need
for stronger IT Governance is stated to ensure
that good, secure IT service will be conceived.
Several actions will be performed
2008 – 2009 in order to fulfil the
Government’s goals. |
References
1. SNAO. IT i verksamhetsutvecklingen: RRV
2002:30
2. SNAO. Webben 1: 2003.
3. SNAO. Vem styr den elektroniska
förvaltningen: 2004:19.
4. SNAO. Project Auditing Information
Security (ten different audit reports):
2005– 2006.
5. SNAO. Government control of
information security work within the
public administration: 2007:10
6. Undall, Bjorn, and Bengt E W Andersson.
”Better managed investment in IT-based
business development,” IntoIT, no. 18
(June 2003).
Bengt E. W. Andersson
Bjorn Undall
Bengt E W Andersson specialises in
auditing the use of IT and information
exchange between Public Administration
bodies. Within the office he has also
been involved in quality assurance and
IT support. He holds a Licentiate of
Philosophy in Information Systems.
Björn Undall’s main audit responsibility is the
effective use of IT in Public Administration. Recently
he has specialised in auditing Information Security
issues. He holds an MBA from the University of
Lund, and has (alas!) unfinished doctoral studies.
35
Dan
AUSTRIA
S OVENI
SL
O
A
SLOVENIA
CROA
OATI
A A
CROATIA
SLOVENIA
An English summary of this report
will be published on the EUROSAI
IT Working Group members
website, but readers can also
contact the Court of Audit on
sloaud@rs-rs.si for a copy.
Audit of IT
system of the Tax
Administration
of the Republic
of Slovenia
Reasons for the introduction of
the audit
The Tax Administration of the Republic
of Slovenia (TARS) in common with other
modern Tax and Revenue Administrations
heavily relies on IT support. The Court of
Audit of the Republic of Slovenia (CoA)
has previously audited TARS and on both
occasions made same request for action,
which has not yet been fulfilled. Another
good reason was that TARS new
leadership indicated willingness to be
audited externally.
Audit approach
Our audit approach was divided into two
main parts. The efficiency part consisted
mainly of an Assessment of efficiency of
TARS IT systems according to version 4 of
CobiT. On the efficiency part, the CoA has
also performed an assessment of the quality
of the information stored in three of TARS
most important IT subsystems
(1 Eppler model). We have tried to evaluate
mainly user experience variables such as
speed, availability, usability and so forth.
The second part – Regularity of TARS
IT system – was quite narrow and only
consisted of compliance and error checking.
36
1
The findings
Our auditors have, together with auditees
staff, performed an evaluation of all 34 IT
processes across four domains. The target
for average grade of all 34 processes that
we agreed with the auditee was to be three.
This desired level was missed by 0.7 with
only two processes reaching a score above
3 and additional five with a grade 3. CoA has
made several recommendations for the most
critical processes.
The assessment of the quality of the
information in three most important
applications (individual and businesses tax,
VAT system, Taxation register) was less harsh
but also in that area CoA agreed several less
formal recommendations with the auditee.
The absence of central and integral
bookkeeping evidence, significant number
of errors in Income tax calculations as well
as absence of controls for its detection and
correction were the reasons for a negative
opinion on the regularity part of the audit.
Development after the
publication of the report
Our report has gained significant public
attention and was quite favourably
accepted by the auditee as well as by
the Public Accounts Committee of the
Slovenian Parliament. The response report
issued by auditee was encouraging. TARS
has introduced significant changes in its
operations, their budget and plans were
amended and public procurement process
for new IT system is already in progress.
Eppler Martin J.: Managing Information Quality, Springer-Verlag Berlin, Heideberg, 2003
Conclusion
The Court of Audit of the Republic of
Slovenia intends to continue this type of
measuring of performance of major public IT
systems. Our goal is to be able to benchmark
different auditees and to be able to show to
the public and to the parliament how good
is the service that our publicly financed IT
systems are providing. With this in mind, we
are striving to do our part in achieving our
mission goal – Watching over
Public Money.
For more information please visit our
website: http://www.rs-rs.si. Some material
is also available in English at http://www.
rs-rs.si/rsrs/rsrseng.nsf. |
In accordance with the Slovenian Constitution
the Court of Audit of the Republic of Slovenia is
the highest body for supervising state accounts,
the state budget and all public spending in
Slovenia. The Constitution further provides
that the Court of Audit is independent in the performance of its duties and bound by the
Constitution and law. The Court of Audit Act also defines that the acts with which Court of Audit
exercises its powers of audit cannot be challenged before the courts or other state bodies.
37
North
S
Amsterdam
SWE
ETH.
Copenhagen
Gulf of Bothnia
ERMA
ERMANY
Stockholm
hol
ollm
Berlin
D
OLAND
Baltic Sea
Tallinn
Tal
Ta
l
Rigaa
R
ingrad
Kaliningrad
aliningr
LITH ANIA
LITHU
LITHUANIA
A
Vilniuss
Minsk
BELARUS
Smolensk
Smo
molens
mo
nsk
NE
UKRAINE
Kiev
OMANI
A A
uchares
charestt
Helsinki
EST.
St. Petersb g
LA
LAT.
AT
rsaw
M
Murm
FINLAND
Pskov
P
skov
sk
Novgorod
Lake
Onega
ega
Shall
be blessed?
Arkhang
Konosha
osh
Tver'
Vologda
Yarosla
aY
ros '
roslavl
Moscow
Mosco
ow
Kostroma
Bry
Br
B
Brya
ryans
y k Kaluga
Ive
Ivenovo
ven
ve
T laa
Tu
Vladimi
V
l im r
Orel
Nizhn
Nizhniyyaazan'
R
N
LLip
ip
ies k
Kursk
ur
K rkiv
Kharkiv
TTambo
b
Kotlas
Kirov
LITHUANIA
Blessed
IS Audit in Valstybės kontrolė, the
National Audit Office of Lithuania
“Hey you, you’re a child in
my head
You haven’t walked yet
Your first words have yet to
be said
But I swear you’ll be
blessed”
Recording facts or events, especially when
they are recent, is an unrewarding task.
Firstly, we unconsciously try to determine
that something important happened
much earlier than it actually did. Secondly,
we over-estimate those important facts
or events which may be witnessed by
others. That’s why such stories often reach
prehistoric times, enhancing today’s deeds
with a patina of antiquity, and awarding
us the status of observers, sometimes
even the status of actors of such events.
And more – it’s very difficult to play the
role of objective bystander when you
are stirred up by the events which are
sometimes, somehow related to you.
by Elton John & Bernie Taupin
Therefore – if no one says otherwise
– 1997 shall be remembered as the start
year of information systems audit at the
National Audit Office of Lithuania. In any
case, this was the year when we first uttered
the words “informacinių sistemų auditas”.
We did not know then what we
wanted, but we knew that “we want”.
The first VFM report (value-for-money
audit as a separate audit area was recognised
by the National Audit Office of Lithuania
only in 2001) was titled “Regarding results of
assessment of activities of establishing and
development information systems in terms of
economy, efficiency, and effectiveness”.
The report recommended:
¬
better coordination between IT strategic
plans of ministries and agencies with
the strategic plan of information
society development for Lithuania;
¬
better coordination and control of
IT projects and initiatives, and;
¬
improvement of data exchange
between state institutions.
we want
Audit of Information Technology. Someone
should be the first to say those words
which are a lovely combination to listen
to, to say nothing of their meaning. There
is a possibility (or a privilege?) to control
(or to be responsible for) something
which is important and meaningful.
Looking for the origins, one may
recall the year of 1997 when the INTOSAI
information systems audit material had
been translated into Lithuanian. Today we
can no longer find anyone at the National
Audit Office of Lithuania who can witness
why and how this material appeared, and
why it was translated into Lithuanian; but
we believe that 1997 marked the beginning
of information systems audit. Reasons,
however, are much more instinctive
rather than consciously recognised.
38
2001 was probably the year when we
already knew what we wanted but did not
know how to achieve it. Luckily (although
someone has said that luck is no more
than a result of your own efforts…)
this year heralded a slow but targeted
and sustainable enforcement of IS audit
function. We must pay tribute to those
who were our patient but strict teachers.
In 2001 and 2002, a joint project with
the Swedish International Development
Cooperation Agency (SIDA) allowed us
to organise two IT audit seminars at the
National Audit Office of Lithuania, led by
specialists from the Swedish Riksrevisionen
(the Swedish National Audit Office).
Figure 1: Types of IS audit
Information Systems Audit
Evaluation of Internal Control
Evaluation in terms of 3Es
existing standards has become obvious.
Such a need was not only an external,
but also an internal factor, conditioned by
growth of complex information systems in
the public sector and – subsequently – by
an increase of finance for such systems.
The need to design one’s own IS audit
methodology and to institutionalise the
IS audit function has become stronger.
we can
General
Controls
Application
Controls
System
development
The years of 2002-2005 were also a
successful learning time. This period saw
the First and the Second PHARE projects,
when our experience was shared with
the National Audit Office of the United
Kingdom, and Danish Rigsrevisionen (the
National Audit Office of Denmark).
And of course, we benefited from
training at the International Centre for
Information Systems and Audit offered by
the Supreme Audit Institution of India.
Even minor efforts, provided they are
persistent and targeted, sooner or later
will bring the desired result. One of the
wins worth acknowledging is that the
first auditor of the National Audit Office
of Lithuania has become CISA certified.
The first such in the Lithuanian public sector.
Sometimes small rivers, being close each
to the other, but separated by hills, have
to make a long journey before they come
together. In the same way, the functions of
information technology governance and
information systems audit have risen from
IS Performance Audit
different springs. Despite bringing their
competencies separately, they, against all
the odds, inevitably approached each other.
From our start in 2002, the functions of
information technology governance and
information systems audit were separate.
But, over time, they grew towards each other.
No-one can say if the CobiT methodologies
(or good practices) for IT governance and
audit were firstly applied to IT governance
or IS audit. The EUROSAI Information
Technology Working Group project
“Information technology self-assessment
for supreme audit institutions” started
in autumn of 2002 and the first IT selfassessment seminar happened on 13-17
October 2003, moderated by representatives
of the Court of Audit of the Netherlands.
This was an important event both for
information technology governance
and for information systems audit.
Lithuania joined the European Union in
2004, and the need to carry out specialised
information system audits according the
In February 2006 the IS audit function
became part of the newly established
Department of Information Technology
Management and Audit, and in
October 2007 the function was entrusted
to the Division of Information Systems
Audit, within the same department. This
is a compact unit of a head and four state
auditors, a good place to grow together and
to become stronger, and a good way to help
a lot of financial auditors to become more
competent in auditing the general controls
of straightforward information systems.
In April 2006, the Methodical
Recommendations for Information
Systems Audit, based on INTOSAI Auditing
Standards, and European Implementing
Guidelines for the INTOSAI Auditing
Standards (Guideline No. 22) were approved.
Methodological Recommendations defines
the place of IS audit, as well as its types,
relation with financial and value-formoney audit, scope and methods.
Methodical Recommendations is an
important document, therefore we will focus
on some of its basic principles. Those who
are interested can find the document on our
web site at http://www.vkontrole.lt/en/docs/
IS_Audit_Methodical_Recommendations_
EN.pdf. We hope this material will be useful.
39
Methodical Recommendations
gives the following definition of
Information Systems Audit:
¬
Audit of Information System
general controls
¬
Audit of application controls
¬
Audit of Information System
development controls
¬
Distribution of tasks between different
levels of auditors are as follows:
¬
IS audits performed by generalist
auditors are limited to medium
complexity evaluation of IS general
control and accounting programmes
(e.g., Navision Financials, LABBIS etc.);
¬
IS auditors perform audits of general
control of complex IS (e.g., IS of the
State Social Insurance Fund Board of
the Republic of Lithuania, Customs‘
IS etc.), IS development audits,
and IS performance audits;
¬
IS/IT specialists provide specialised
guidance on particular issues.
Information System performance audit.
The objectives of the different IS
audit types are as follows:
¬
Audit of IS general controls. Evaluate
internal control which covers all
information systems of an organisation.
¬
Audit of application controls. Evaluate
a control related to data input,
processing, protection, and obtaining
in the specific applications (e.g.,
Navision Financials or LABBIS).
¬
Audit of IS development controls.
Evaluate management and
control of IS development from
conception to live running; covering
IS change management;
¬
Objective of IS performance audit.
Evaluate issues related to IS in terms of
efficiency, economy, and effectiveness.
In terms of IS audit, INTOSAI distinguishes
three levels of auditors (implementation
of these levels in the National Audit Office
of Lithuania is presented in Figure 2.):
¬
Public auditors conducting
financial and performance audits
(hereinafter – generalist auditors),
¬
IS auditors,
¬
IS/IT specialists.
During financial or performance audits
generalist auditors may ask for help from
IS auditors or IS/IT specialists. In such
cases the Audit Department Director
(Deputy Director) applies to the Head
of the structural unit of the NAOL which
performs IS audits. If necessary, an external
IT/IS specialist can be brought in.
Having performed IS audits, generalist
auditors present their evaluation of IS
general controls to the structural unit of
the NAOL which performs IS audits.
Two illustrious value-for-money
audits were carried out in 2006 and 2007,
aimed at IT governance in governmental
(supra-ministerial) institutions which made
many recommendations for the government
to assure proper governance of the IT
function. Among them, recommendations
on improvement of management structures
of e-government projects as well as on their
comprehensive quality control in terms of
efficiency, economy, and effectiveness.
We are active participants in the
project “Information technology audit
function self-assessment for supreme audit
institutions” and the first pilot seminar
in Vilnius on May 22-23, 2007, launched
by the EUROSAI Information Technology
Working Group project. A good basis for
future action, “we shall do” indeed! |
we shall
The existence of Methodical
Recommendations, and the distribution of
tasks between IS auditors and generalist
auditors have allowed the last-mentioned
to perform 76 general control evaluations
of simple information systems in 2006. At
the same time, IS auditors carried out six
general control evaluations of complex
information systems, a good distribution of
tasks and a good use of IS auditors’ potential.
Figure 2: Division of functions when performing IS audit
IT/IS Specialist
IS Auditor
Specialists
from internal IT
Department
Helps to transfer data from
the IS of audited entity to
computerised audit tools
External IS/IT
Specialists
40
Generalist Auditor
Financial auditors use
computerised audit tools,
carry out risk evaluation
in non-complicated IS,
Evaluates general controls
having encountered
of information system
problems relating to clients
Helps to perform audits of
IS, consult IS auditors for
application software for financial
more detailed analysis
and performance auditors
Performance auditors use
Participates in evaluating
computerised audit tools,
of IS from the point of view
participate in analysing
of economy, effectiveness
correctness, reliability and
and efficiency
comprehensiveness of
Prepares IS methodologies
management information
and provides training
and evaluate IS from the
Provides information to
point of view of economy,
IT Department about
effectiveness and efficiency
good IT governance /
management practice
Dainius Jakimavičius
Dainius Jakimavičius graduated from Vilnius University
in 1983. He became a Doctor of Mathematics in 1993.
He has worked in the Lithuanian National Audit
Office since 2001. He has been successively Head of
Information Technology Division (2001-2002), Director
of Information Management Department (2002-2004),
Director of Information Technology Department
(2004-2006) and currently Director of Information
Technology Management and Audit Department.
41
mu
Juzno
Sac a s
O C E A N
sk
S
Sapporo
Sea of
Japan
n
S
Sendai
(East Sea)
n
H
NORTH
A
KOREA
yo
Tokyo
JAPAN
Pyongyang
Fu
eoul
SOUTH
KOREA
Yellow
Sea
SShanghai
st China
JAPAN
Auditees
The auditees were: Cabinet;
Cabinet Office; Ministry
of Internal Affairs and
Communications; Ministry
of Justice; Ministry of
Foreign Affairs; Ministry
of Finance; Ministry of
Education, Culture, Sports,
Science and Technology;
Ministry of Health, Labour
and Welfare; Ministry
of Agriculture, Forestry
and Fisheries; Ministry
of Economy, Trade and
Industry; Ministry of
Land, Infrastructure and
Transport; Ministry of the
Environment; the Diet;
Courts; and the Board of
Audit of Japan
r
ce
n
Ca
of
c
i
op
Tr
Audit of
Computer
Systems used
by the Japanese
Government
Topics Covered
1. Outsourcing contracts concluded by
the Cabinet Office and Ministries with
system integrators (SI) including NTT Data
Corporation.
2. Competitiveness and economy of
maintenance and operation contracts.
3. Use of major systems.
4. Management of information security.
5. Present situation of the systems
(including legacy systems) for which
operation and system optimisation plans
are to be made, based on the Programme
for Building e-Government and the
measures being implemented towards
the optimisation.
6. Verification of the final accounts in
consideration of the above.
Results of the Audit
Outsourcing contracts concluded by the
Cabinet Office and Ministries with System
Integrators including NTT Data Corporation
a. Outline of information systemrelated contracts concluded by the
national government
The Board of Audit of Japan analysed
the national government’s information
system-related payment in the fiscal year
2004. There were 6,475 contracts and 477.3
billion yen concerning 77 operations and
systems covered by the optimisation plans
in the administrative agencies and the Diet,
Courts, and the Board of Audit of Japan itself,
for which the payment amount exceeded
1 million yen.
b. Contracting parties
As for the payment to the contracting parties
of the 6,475 contracts, the payment to NTT
Data Corporation was the largest, amounting
to 173 billion yen and accounting
42
for 36.2% of the total. The payment to the
top five contractors accounted for 65.4% of
the total payment amount.
c. Tendering procedures
Of the 6,475 contracts, the tendering
procedures for 2,873 contracts for each
of which 3 million or more was paid
(amounting to 473.2 billion yen in total) are
that 80.8% of the contracts and 96.3% of the
total payment were single tendered. Most
of the national government’s information
system-related contracts are awarded using
single tendering procedures.
Competitiveness and economy
of the maintenance and
management contracts
Of the 2,873 contracts worth three million
yen or more, the Board of Audit of Japan
examined the competitiveness and
economic efficiency of 492 maintenance and
management contracts (worth 36.6 billion
yen) concluded by the internal departments
of the Ministries and Agencies.
a. Competitiveness for
maintenance and management
contracts
Of the 492 contracts, those awarded by
competitive tendering procedures accounted
for 8.1% by number and 3.9% by value,
while those awarded by single tendering
procedures accounted for 91.8% by number
and 96.0% by value. Thus the percentage of
contracts awarded by competitive tendering
procedures is low. For 458 contracts
(excluding unit-price contracts from the
492 contracts), the average ratios of the
successful bid price to the planned price by
type of tendering procedures were 81.9% of
competitive tendering (94.3% of one bidder
and 60.9% of multiple bidders) and 97.4% of
single tendering.
For 168 contracts (30 contracts of competitive
tendering and 138 contracts of single
tendering) selected from the 492 contracts,
the following items of written specifications,
which would be important to increase
competitiveness in tendering, were more
frequently described in competitive tendering
than in single tendering: [1] work amount
by work item and data on the occurrence
of troubles; [2] scope of responsibility;
and [3] system component items.
b. Calculation of the planned
prices for maintenance and
management contracts
From the 492 maintenance and management
contracts, the Board of Audit of Japan first
selected the contracts under common
operational items ([1] system monitoring;
[2] preventive maintenance; [3] responses to
inquiries from the officials in charge of the
system; [4] troubleshooting; and [5] system
operation). Then the Board extracted 112
contracts. As for the calculation methods of
the planned prices, 26 Ministries and Agencies
that awarded these 112 contracts have no
manuals based on which their planned prices
shall be calculated. For the unit personnel cost
(man-months cost in yen) of system engineers
(SE) and others, as for the materials based on
which the unit cost was determined for the
112 contracts, the documents most frequently
referred to are the written estimates submitted
by system integrators. Also, the unit personnel
cost paid for the same operation based on the
same reference material varies by contract.
43
As for the verification of the
appropriateness of the SE-related price
estimates in the planned prices, the
appropriateness was not specially verified
for 29 contracts (25.8%) among the 112
contracts and the appropriateness was
verified for 59 contracts (52.6%) by service
reports. (The appropriateness was verified for
14 of 59 contracts by calculation of the unit
hours required for each operation.)
Use of major systems
a. Use of electronic application
systems
The Board of Audit of Japan conducted
audits on the following electronic
application systems managed and operated
by the internal departments of the
Ministries and Agencies: 16 general-purpose
systems of 16 Ministries and Agencies; 25
special-purpose systems of 12 Ministries
and Agencies. Thus the Board examined
a total of 41 systems of 20 Ministries and
Agencies, for which they had paid a total
of 32.9 billion yen in fiscal years 2003 and
2004. As at the end of September 2005,
electronic applications can be made for as
many as 14,354 procedures. General-purpose
systems were used for 12,899 procedures
and special-purpose ones for 1,455. Of the
procedures for which electronic applications
are possible as of the end of fiscal 2004,
the percentage of procedures for which
the total number of applications (number
of electronic applications + number of
written applications) was zero account for
52.4% of the procedures to be processed by
general-purpose systems and 23.7% of the
procedures to be processed by specialpurpose systems.
The percentage of electronic applications
processed by the general-purpose was
0.02% and the percentage of electronic
application processed by special-purpose
systems was 5.57%. 0.94% is in the total
number of applications in fiscal 2004.
b. Use of electronic bidding
systems
The Board of Audit of Japan conducted
audits on 12 electronic bidding systems
managed and operated by the internal
departments of 12 Ministries and Agencies.
The payment made in relation to these
systems in fiscal years 2003 and 2004
amounted to 4.6 billion yen.
For the use of electronic bidding systems
from fiscal year 2003 to fiscal year 2005
(to September 2005), the percentage of
electronic bidding (number of contracts for
which whole or part of applicants made a
bid through the electronic biding systems
divided by the number of contracts for
which electronic applications could be
made) remained relatively lower for goods
and services than for construction.
44
Management system for
information security
a. Information security measures
The Audit Board examined the information
security measures implemented by the
internal departments of Ministries and
Agencies as of the end of October 2005.
As for the procedures to enter and exit the
server room, 10.8% of the departments
“have no application procedures and do
not keep the entry/exit records at all.” As
for the monitoring of LAN devices by the
use of network monitoring equipment,
6.8% of the departments do not conduct
monitoring on any possible attacks to the
LAN. For the backup of data stored in a
variety of systems installed at the server
room, there is no backup data or the
backup data is stored only in the server
room for nearly half of the data (45.7%). As
for the accessibility to data folders of the
divisions of the internal departments, they
are “readable” at 5.7% of the departments.
3.4% of the departments have virus
definition files updated by users manually.
As for the use of privately owned PCs at
offices, 59.3% of the departments “do not
prohibit the use.” Regarding the connection
of privately owned PCs to the LAN, many of
the departments “prohibit the connection”,
but at many of these departments, “users
can connect their private PCs to the LAN if
they configure some settings”.
b. Management System for
Information Security
Of 25 Ministries and Agencies (those
excluding the eternal organs under control
of the Ministries and others from the
Ministries and Agencies), 23 had formulated
information security policies but only seven
of them conducted risk assessment when
they formulated their initial policies. Only
12 Ministries and Agencies had provisions
for the establishment of audit teams, of
which only four actually had those teams
in place. As for written procedures to
implement information security, three had
not created any such manuals. One of the
Ministries and Agencies had its audit team
“check the compliance with the Policies,”
four conducted “self-examination,” eight
conducted “information security audits,” and
17 conducted “vulnerability assessments”.
Present situation of the systems
(including legacy systems) for
which operation and system
optimisation plans are to be made
based on the Programme for
Building e-Government and the
measures being implemented
towards the optimisation
a. Present situation concerning the
systems for which operation and
system optimisation plans are to
be made
As of the end of June 2005, a total of 77
operations and systems in the Ministries
and Agencies were included in the target of
optimisation plans. Their management cost
came to 465.3 billion yen in fiscal year 2004.
Of the 77 operations and systems, 36
are legacy systems used at 16 Ministries and
Agencies. For the management of these
legacy systems, the Ministries and Agencies
paid the contract amount of 345.8 billion
yen in fiscal year 2004, which accounts
for 74.3% of the amount paid related to
the 77 operations and systems. Of data
communications service contracts, the
service fees reached or exceeded 100 million
yen for nine contracts in fiscal year 2004 and
a total of 157.6 billion yen was paid for these
contracts. As of the end of fiscal 2004, the
Remaining Debt concerning the nine data
communication services contracts totalled
164.2 billion yen.
b. Formulation of the operation
and system optimisation plans
For the 77 operations and systems for which
optimisation plans were to be made by the
end of fiscal year 2005, a total of 7.8 billion
yen was paid as expense for commission.
c. Cost reduction effect and
problems described in the
optimisation plans
The Board of Audit of Japan estimated the
development cost. Some operations and
systems of the development cost will be
recovered as a result of reduced management
cost within four years, but the others will not
be recovered within four years.
d. Problems to solve to ensure
the appropriateness of the
optimisation plans
One of the outcomes of the optimisation
plans is DFDs (standard document formats
are created based on data flow diagrams).
There were as many as 831 inconsistencies
found in the description of information
flow in 47 of 66 optimisation plans. Some of
common operations and systems still need
to be coordinated.
Verification of the final
accounts based in
consideration of the above
(i) The Board of Audit of Japan checked the
contracts for 77 operations and systems
of the administrative agencies for which
optimisation plans were to be made and
which accounted for most part of their
information system-related budget as
well as the contracts worth of 1 million
yen or more concluded by the Diet,
courts and the Board of Audit of Japan
itself. As a result, it was revealed that the
information system-related payment
made in fiscal year 2004 totalled as much
as 477.3 billion yen.
As for the contracting parties, payments
to the top five companies accounted
for 65.4% of the total. For tendering
procedures, the percentage of competitive
tendering was low.
(ii) As for maintenance and management
contracts, those awarded by single
tendering procedures accounted
for 91.8% in the number and 96.0%
in the amount, demonstrating low
competitiveness in tendering. The
rate of the Successful Bid Price was
higher for contracts awarded by single
tendering than for those awarded by
competitive tendering, in particular by
multiple bidding.
The planned unit personnel cost greatly
varies by contract. Such verification and
reflection of the results were not
sufficiently conducted.
(iii) As for the procedures for which
electronic applications can be made
through electronic application-related
systems, there were no online or written
applications filed in fiscal year 2004 for
52.4% and 23.7% of such procedures for
general-purpose systems and specialpurpose systems, respectively. The total
percentage of electronic applications was
low as 0.94% for the general-purpose and
special-purpose systems.
(iv) For the information security measures
implemented by the internal
departments of the Ministries and
Agencies as of the end of October
2005, there are the deficiency of
data-and privately owned PC-related
security measures. The management of
information security is inadequate.
|
(v) The cost of managing 77 operations
and systems for which optimisation
plans would be formulated came to
465.3 billion yen in fiscal year 2004 which
the cost of managing the 36 legacy
systems amounted to 345.8 billion yen
(74.3%). As for the cost reduction effect
estimated in the optimisation plans, they
need to be improved and reviewed to
attend the effect. The Board of Audit of
Japan found a lot of inconsistencies in
the DFDs to be included in the
optimisation plans.
Opinions about the Audit Results
The following measures should be
implemented to ensure the economical,
efficient, and effective implementation
of the national government’s information
system-related budget.
1. to improve the competitiveness and
transparency of the contracts and to
improve the rationality of the calculation
of planned prices
2. to promote the use of the electronic
application-related systems and thereby
increasing the convenience of people
3. to enhance the security measures and
to improve the management system for
information security
4. to implement their operation and system
optimisation plans while to ensure that
the plans respond to the changes of
the situation.
The Board of Audit of Japan will keep its
eyes on the movement of the government
towards the implementation of the
optimisation plans and conduct
multifaceted audits on the government’s
computer systems.
45
Ulaangom
Amur
Hailar
Hulun Nur
Ulaanbaatar
Hovd
Choybalsan
Irtysh
Qiqihar
Bayanhongor
Altay
Jixi
Harbin
M O N G O L I A
y
Jilin
Changchun
Urumqi
Vladivosto
Dalandzadagad
Hami
Fuxin
Baotou
Huang Ha
Yumen
Zhangjiakou
Datong
Beijing
Tangshan
Hu
Xian
Shiquan
Jinsha
R.
Hefei
Yangtze
Huangshi
Nanchang
Str
.
Shaoguan
Xiamen
Guangzhou
Nanning
Chittagong
Sittwe
MYANMAR
VIETNAM
LAOS
Taipei
y
u
k
y
Taiwan
Shantou
Kaohsiung
Zhanjiang
Hai Phong
Gulf of
Tonkin
Chiang Mai
o f
Salween R.
B a y
m
Vinh
Vientiane
Nong Khai
S o u t h
C h i n a
S e a Laoag
s
l
a
n
Kume
u Shima
d
s
Tokuno Shima
kino Erabu Shima
O
kinawa
O
Miyako Retto
Ishigaki Shima
Iriomote
Jima
R
Hong Kong
Hanoi
Mekong
Amami OShim a
I
Kunming
Mandalay
S e a
Wenzhou
Fuzhou
Dhaka
Monywa
C h i n a
Pingxiang
Khulna
kata
E a s t
Shanghai
osa
Imphal
Yueyang
Kyus
Wuxi
Wuhu
Huzhou
Jiaxing
Hangzhou
Ningbo
Jingdezhen
Shang Rao
Fo
rm
Salween R.
es
Pusan
Changsha
Guiyang
Dukou
Mekong
maputra
Brah
Taegu
Kita-kyushu
Fukuoka
tze
ng
Ya
Wuhan
Chongqing
Jin
sh
a R.
Zigong
Qingdao
Huainan
Nanjing
Mianyang
Chengdu
SOUTH
KOREA
Taejeon
Incheon
Yellow Sea
Zaozhuang
Kaifeng
Xuzhou
Huaibei
Luoyang
Thimphu
Ha
Taian
Lanzhou
Lhasa
Seoul
ang
Jinan
Golmud
BHUTAN
Dalian
Shijiazhuang
Handan
Hu
an
g Ha
Benxi
NORTH
Yingkou
Feng Cheng KOREA
Dandong
Pyongyang
Tianjin
Taiyuan
Xining
Salween
Jinzhou
Hohhot
Yinchuan
C H I N A
BANGLADESH
Liaoyuan
Fushun
PHILIPPINES
LIPPINESS
THAILAND
on
CHINA
In December 2007 the National
Audit Office of China formulated
a pre-audit investigation
guideline for IT audit, as one
part of the Chinese Government
Auditing Standards System.
P a
i f i c
O
e a n
CNAO’s
Pre-audit
Investigation
Guideline for
IT Audit
The National Audit Office of the People’s
Republic of China (CNAO) attaches great
importance to the pre-audit investigations
of IT audit projects. Prior to preparing audit
implementation programmes, according to
the nature and scale of the audit project, the
audit team is required to arrange competent
staff to know about the information of
auditees. Therefore CNAO has formulated this
Pre-audit Investigation Guideline for IT Audit.
This Guideline can be used in pre-audit
investigations where:
¬
The auditee has computerised
its accounting or other main
business systems;
¬
The audit team are carrying out an
Information Systems, E-Governance
or IT performance audit.
The CNAO requires auditors to pay attention
to the substantial changes brought to the
audit institutions and the auditees by the
application of information technologies. So
the target of the pre-audit investigations on
IT Audit is to make the audit implementation
programs prepared by the audit teams and
audit institutions meet the needs of audit
in an IT environment. According to the
Guideline, auditors involved in the pre-audit
investigations should have appropriate
IT knowledge and skills. If necessary,
professional IT staff from CNAO’s IT Centre or
external IT specialists could be invited to join
the audit team.
Pre-audit investigations can be
implemented through consultation, group
interview, questionnaires and surveys,
information/data inquiry, on-the-spot
inspection and visiting related organisations.
46
Through the pre-audit investigations,
auditors should obtain basic information
as follows:
Firstly, the information systems used
by auditees, including: the method
and time of acquisition, operating
system, database management system,
application software versions, hardware
configuration, data processing flow,
interaction with other information
systems, data output type and format,
system controls and security policies.
Tests could be carried out in the
Auditee’s information system during
the pre-audit investigation, only when
the normal operation of the target
information system is ensured.
Secondly, Auditee’s electronic data,
including: data storage medium; data
volume measured by gigabyte (GB); the
compliance level of output data to the
Chinese National Standard, whether the
data could be collected successfully by
audit software such as Auditor Office
(AO); the capability of auditees to
support the auditors’ data-collecting
processes; the preliminary check for the
authenticity and applicability of the data
during the pre-audit investigation.
Thirdly, the dependence of auditees’
business flow on information
technologies, including the degree of
impact on the continual operation of
auditees; the popularity of information
systems. Auditors can review the
dependence indicators by sampling
during the pre-audit investigation.
Fourthly, IS management and
management styles, including: the
legal requirements for the IS in use;
the position of the IT Centre in the
organisation chart; segregation of
duties between IS managers and users;
the setting of major control points
and posts. Auditors can review the
management structures by sampling
during pre-audit investigation.
Fifthly, the environment under which
the IT audit is carried out, including:
the facilities and network environment
provided by auditees, the equipment and
facilities that audit teams should prepare;
the software supplied by auditees,
the software to be prepared by the
audit teams; assessment of the security
impact to both auditors and auditees
under the IT environment of auditees.
The process of pre-audit investigations and
information collected should inform:
¬
Audit objectives;
¬
Audit contents and priorities;
¬
Audit items, which could have significant
influence on audit objectives;
¬
Significance level and audit risks;
¬
Organisational manners and working
methods under IT environment;
¬
Computer equipment and environment
necessary for audit and their solutions;
¬
The number and skills of IT professional
staff needed in the audit team;
¬
Estimated audit working
time span and budget.
If, based on the preliminary result of
pre-audit investigation, the audit team
believes auditee’s information system has
weakness and may significantly influence the
authenticity and integrity of the electronic
data, the team can recommend that an
information system review be added to the
audit implementation programmes. |
Wang Zhiyu, Director General, IT Centre of CNAO
Wang graduated from Zhengzhou University of China
and was awarded a Bachelor degree in Economics
in 1981. He was appointed as the Director General
of CNAO’s IT Centre in 1999. As a CIO, at present he
is responsible for the implementation of Golden
Auditing Project. Because of his excellent work, he
was awarded the Prize of Outstanding Contributor
for Promoting IT Application in China in 2004.
47
The INTOSAI information
technology journal
it
© National Audit Office 2008 | Design
and production by NAO Marketing &
Communications Centre | DG Ref: 8266RD |
Printed by Heronsgate
Printed on Greencoat paper. Greencoat is
produced using 80% recycled fibre and 20%
virgin TCF pulp from sustainable forests.
www.intosaiitaudit.org