Assessing Credit Rating Agencies: Quis Aestimat Ipsos Aestimatores?

Who rates the raters themselves?

Just as Juvenal wondered who guards the guards, the international financial community wonders who rates whom. The spotlight has been on the international auditing and accounting profession, but investor credit rating agencies share the limelight. In some ways, governance of credit rating agencies is a thornier problem than the governance of auditors.

At first glance, there is little to concern a free marketeer. The only regulatory oversight of note for rating agencies is the NRSRO (nationally recognized statistical rating organization) status conferred by the SEC. Four credit rating agencies have NRSRO status - Moody's, Standard & Poor's, the Fitch Group and Dominion Bond Rating Service. There are a host of smaller credit rating agencies typically focused on non-USA organizations or with specialist niches, e.g. Capital Intelligence (international banks), Japan Credit Rating Agency, ICRA (India), PEFINDO (Indonesia).

A number of agencies rate corporate or consumer creditworthiness for non-investors, e.g. Dun & Bradstreet or Equifax, and other agencies rate aspects of organizations than credit, e.g. SERM Rating Agency (social and environmental ratings). Further, there are any number of commercial organizations that grade other organizations' adherence to best practices in a variety of fields, e.g. SGS, Det Norske Veritas, Bureau Veritas, Lloyd's Register, as well as not-for-profit organizations. There are large numbers of standard-setters, both commercial and not-for-profit, from the ISO network of 140 governmental standards agencies (all types of technical standards) to the Marine Stewardship Council (sustainable fishing). Sometimes the standard-setters also assess/certify/rate, e.g. the British Standards Institute. Rating other organizations is clearly a flourishing industry.

In such a competitive market investor credit rating agencies must be adding value for their customers, yet the financial community perceives that credit ratings are controlled by a duopoly - Moody's and Standard & Poor's - or at best a triumvirate - if Fitch is rated in the top group. "There's too much power in too few hands," said Dietrich Jahn, a German ministry official who focuses on banking [International Herald Tribune, "Germans Call for Local Rating Agency", 4 March 2003]. The financial community also questions the usefulness and quality of ratings themselves.

Who are the stakeholders?

Credit rating agencies typically rate debt - government (sovereign, agencies), corporate (banks, debt issuers) and special purpose vehicles or issues. "A rating simply helps investors determine the relative likelihood that they might lose money on a given fixed-income investment" [Moody's]. From this, it sounds as if credit rating agencies act on behalf of investors. However, credit rating agencies receive relatively little money from investors for research or publications; far and away the bulk of credit agencies' income is from the issuing organizations. The agencies are careful to position themselves as offering only an "opinion". They take no responsibility for the use of their ratings. They offer no recourse if their rating fails to identify problems with debt repayment, e.g. compensation. In fact, in conversation, three rating agencies concurred with the author that if they had to rely on investors paying for their services, they would go out of business.

Rating agencies can perform a crude form of outsourcing for banks. Rather than have everyone find, churn and analyse the same information internally, they can all rely on the ratings in the first instance. Z/Yen goes further, the ratings can form an important benchmark against which internally produced ratings can be contrasted. A bank's confidence in the divergence of its ratings from the credit rating agencies measures its perception of competitive advantage.

Credit ratings are used out of context through no fault of the credit rating agencies. Organizations bandy about the familiar AAA through BB (Standard & Poor's scale) as if they apply to the organizations and not just organizations' debts. This is hardly surprising. The organizations have paid handsomely for a rating that implies some form of quality and it is up to the user to ascertain that the rating was provided against the stricter scope of investment debt - caveat quicumque. Credit ratings are seals of approval for marketing purposes.

On the other hand, credit ratings are used for investor protection. Many funds are restricted to investments with higher ratings. Some of these restrictions are voluntary and designed to make the funds more attractive to risk-averse investors. Some of these restrictions are regulatory, for instance placed on civil service or public employee pension funds by local or national governments. USA government restrictions typically require that the credit rating agency have NRSRO status, until recently held only by USA rating agencies and even now only held by four rating agencies (two from the USA - Moody's and Standard & Poor's, one European - Fimalac through the acquisition of Fitch, one Canadian - Dominion Bond Rating Service awarded in February 2003).

Changes in credit ratings affect, sometimes quite severely, the value of debt. Depending on the direction of the change in value, credit rating changes can benefit or harm investors in fixed-income securities, taxpayers (through the cost of government debt), organizations' direct cost of debt and holders of equities. These widespread effects lead to calls for regulation of credit rating agencies. For instance, there have been calls for more frequent (and less frequent) reporting, more structured (and less structured) reporting timetables, tighter (or broader) qualifications to be a rating agency employee, audits of rating methodologies, investor compensation schemes, etc. Arguably, investor protection underlay the need for the NRSRO designation, not restrictive buy-USA intentions.

Outside the USA, organizations complain that their asset weightings, gearing, use of land, local taxation structures or governance bodies put them at a disadvantage. They are compared against a USA benchmark regardless of whether their local structures work for them. "We have to ask ourselves whether the ratings agencies are really sensitive to German business practices, or whether they only operate on the basis of Anglo-American business principles" Rainer Wend, Chairman of the German parliament's economic committee [FOW, April 2003]. This perceived bias by ratings agencies for USA-comparable structures is alleged to cause an over-weighting in USA investments by fund managers.

Nevertheless, credit ratings also seem restrictive to fund managers. For issuers and investors seeking a means of breaking the restrictions, there are numerous ways to arbitrage ratings, ranging from bundling different assets to slicing layers of credit ratings, with the lowest layer left un-rated. There are also insurers who pay out if issuers of certain credit ratings fail to pay, for instance Financial Security Assurance which provides financial guaranty insurance on principal and interest for asset-backed securities, municipal bonds and other structured obligations.

Surely this is evidence of a competitive market. "…an even simpler answer is that it is a game of rating arbitrage. If the ratings did not carry such weight with regulators, and in fund-management mandates, neither would the shifting of assets from one rating grade to another. 'How can you repackage and apparently add such value?' asks Con Keating of the Finance Development Centre in London" [The Economist, "Badly Overrated", 16 May 2002]. There is certainly a problem identifying the stakeholders. Are they issuers, investors, fund managers, the general public? The issuers pay. If credit ratings were compared to audit firms, credit rating agencies would have a 'duty of care' to, at the very least, the shareholders of the organizations they rate, as well as, perhaps, taxpayers for government issue ratings.

Deeper issues

A number of critics of credit rating agencies, for instance Andrew Fight (author of The Ratings Game, John Wiley & Sons, 2001), "suggest the need to rein in the power and market dominance of these agencies" [page 247]. These critics look beneath the surface of the market to other disturbing issues.

First and foremost - are credit rating agencies any good? The "relative likelihood" of loss can be correlated with actual loss. Naturally, ratings agencies provide a lot of PR showing correlations between poorer ratings and defaults. However, anecdotal evidence points to numerous discrepancies between credit spreads and credit ratings. Most of the positive predictive evidence is based on USA bonds where credit rating agencies have a long history, but critics point to overseas bank default rates (less than predicted) or overseas corporate default rates (ditto) or newer, exotic investments (higher than predicted).

Critics wonder how crude rating categories from the beginning of the last century can begin to correlate with the fine credit distinctions of modern quantitative finance. The "repackaging game" would not be possible unless ratings provided large arbitrage opportunities, either by being poor predictors or by creating large valuation changes among rating categories and between rated and unrated.

Financial analysts point to the fact that selective coverage (larger rating agencies do not provide consistent, universal coverage at the level of detail of, say, all German banks or all large Korean companies) means that ratings turn out to be of little use in detailed financial analysis. Z/Yen has conducted predictive evaluations for clients where ratings of UK insurers, UK investment banks and overseas banks had no predictive power. Finally, some analysts note the importance of timescales. They observe that many ratings are downgraded just before a known problem crystallises, i.e. purportedly predictive when looking at historical data, but only after the entire market knew there was a problem, as well as exacerbating the problem. These analysts find little evidence of agencies' ratings being useful leading indicators.

There is also a fair bit of ammunition for critics who claim that rating agencies can be compared, in certain cases, to protection rackets. They cite the practice of unsolicited ratings whereby the credit rating agencies rate an organisation 'in the public interest' and then appear to improve the rating subsequent to being paid by the organisation 'to better understand the background'. The agencies state that this practice was stopped a few years ago. Some of the critics won't let them win - sure the practice has been halted, but only because the credit rating agencies 'control the 'hood' (neighbourhood)'. This seems unfair, but at the time the practice called into question whose interests were being served. The organization's? Probably not, just uncalled for management diversion. Investors? Well some were private companies or government organizations. Public interest? Well, with no demand and no one paying for the original rating it seems altruistic to the point of incredible. It also raises the question of consistency and quality. If there is a free service and it is the same quality, then why should organizations pay in order to be better understood?

Other critics attack perceived conflicts of interest in the ownership of the agencies, to the point of claiming that the agencies should be government entities. It is difficult to see this point. None of the agencies appear to be unduly influenced by their owners and no claims seem to have stood up. If there are conflicts of interest, they are probably within the overall process or the profit-making motivation. Other critics complain that the agencies' employees are unqualified for an unspecified task. This too seems a bit rich. The rating task is a multi-disciplinary one - looking at finances sure, but also markets, consumers, politics, etc. The key skill is assembling diverse types of data, information and knowledge into an overall assessment.

Quid commendas?

Many of these criticisms are old. They come to the forefront for at least a couple of reasons. Firstly, the increased interest in oversight given the failings of the financial system to protect investments in Enron et al. Secondly, the increasing number of propositions to rely on rating agencies, for instance as the basis for risk assessments for banks under Basel II, have led many to invoke Marilyn Strathern's restatement of Goodhart's law - "When a measure becomes a target, it ceases to be a good measure." Others are just worried about the measure as it stands.

So what recommendations might be made to satisfy some of the critics? There are two opposing schools of thought. Lots of regulation, oversight, qualifications and government specification, or, improve the free market. The first school is unlikely to provide results for almost too many reasons to mention, but include the glacial pace of international financial reforms, the lack of agreement on the roles of the agencies, the obscurity of the qualifications needed for an opaque process and the difficulty in getting the user of information to pay rather than the provider.

So what recommendations can be made to improve the market? Of the many ideas floating around, two seem simple, sensible and easy-to-enact. Firstly, NRSRO status should be much more freely given, or even abolished. The SEC is investigating codifying the current NRSRO application process and setting out a list of criteria used by it when reviewing NRSRO applications. However, one has to question why the NRSRO process exists at all. It has only approved four organizations in decades, so is it needed? The second idea is a classic first response to most perceived market imperfections - transparency. If companies and public organizations are paying for credit ratings, then they can declare the payment in their accounts as they do for their audits. This initial response may well be enough to shed enough light that people either understand, or see as insignificant, the rating agency process.

The rating agencies are in for tough criticism ahead as their power is perceived to grow. The abolition, or massive opening, of the NRSRO status may have little effect given the strength and reach of brands and networks. It would, though, remove the one big criticism that agencies use government status to keep out new entrants. It would also reflect well on the SEC to remove an unnecessary qualification. The publication of the rating cost is more problematic and raises issues of commercial confidence. However, as the agencies move to the center stage, they must either accept more regulation or more openness. As business people, they must know that choosing the latter is the right course in the long term, both selfishly and in the public interest.

Michael Mainelli, FCCA, originally did aerospace and computing research, before stooping to finance. Michael was a partner in a large international accountancy practice for seven years before a spell as Corporate Development Director of Europe's largest R&D organisation, the UK's Defence Evaluation and Research Agency, and becoming a director of Z/Yen (Michael_Mainelli@zyen.com).

Z/Yen Limited is a risk/reward management firm working to improve business performance through better decisions. Z/Yen undertakes strategy, finance, systems, marketing and organisational projects in a wide variety of fields (www.zyen.com), such as recent projects developing a stochastic risk/reward prediction engine and benchmarking of transaction costs across 25 European investment banks. Michael's humorous risk/reward management novel, "Clean Business Cuisine: Now and Z/Yen", written with Ian Harris, was published in 2000; it was a Sunday Times Book of the Week and even Accountancy Age described it as "surprisingly funny considering it is written by a couple of accountants".

[A version of this article originally appeared in "Assessing Credit Rating Agencies: Quis Aestimat Ipsos Aestimatores?", Balance Sheet, The Michael Mainelli Column, Volume 11, Number 3, (August 2003) pages 55-58.]