A Unified Methodology for Pre-Silicon Verification and Post-Silicon Validation

    Allon Adir, Shady Copty, Shimon Landa, Charles Meissner, John Schumann
    Amir Nahir, Gil Shurek, Avi Ziv IBM Server and Technology Group
    IBM Research - Haifa, Israel Austin, TX, USA
 Email: {adir, shady, shimonl, nahir, shurek, aziv}@il.ibm.com Email: {cmeissner, johnschu}@us.ibm.com

Abstract—The growing importance of post-silicon validation in pre-silicon. This creates an increasing need to bridge the gap ensuring functional correctness of high-end designs increases the between these two domains by sharing methodologies and need for synergy between the pre-silicon verification and post- technologies and building a bridge allowing easier integration silicon validation. We propose a unified functional verification methodology for the pre- and post-silicon domains. This method- between the domains. ology is based on a common verification plan and similar lan- We propose a unified methodology for pre- and post-silicon guages for test-templates and coverage models. Implementation verification. The methodology extends the well-established of the methodology requires a user-directable stimuli generation coverage-driven verification (CDV) methodology [8] to the tool for the post-silicon domain. We analyze the requirements post-silicon verification domain. CDV is based on three main for such a tool and the differences between it and its pre-silicon counterpart. Based on these requirements, we implemented a components: A verification plan comprising a large set of tool called Threadmill and used it in the verification of the IBM features in the Design Under Verification (DUV) that need POWER7 processor chip with encouraging results. to be verified; random stimuli generators directed towards I. INTRODUCTION the verification goals using test-templates [9] (i.e., general specifications of the desired test structure and properties); The size and complexity of modern hardware systems have and coverage analysis tools [10] that detect the occurrence of turned the functional verification of these systems into a events in the verification plan and provide feedback regarding mammoth task [1]. Verifying such systems involves tens or the state and progress of the verification process. hundreds of person years and requires the compute power of The unified methodology calls for pre- and post-silicon thousands of workstations. But even with all this effort, it is to share the same verification plan, use similar languages virtually impossible to eliminate all bugs in the design before to define test-templates for both domains, and use the same it tapes-out. In fact, statistics show that close to 50% of chips coverage models to measure the status and progress of the ver- require additional unplanned tape-outs because of functional ification process. The methodology provides many advantages bugs. Moreover, in many cases, project plans call for several to users. It provides commonality and enables sharing of effort planned tape-outs at intermediate stages of the project before in the creation of the verification plan. It can also facilitate the final release of the system. As a result, an implementation mutual feedback between the two domains, for example, in of the system on silicon running at real-time speed is available. the activity of bug hunting. This silicon is used, among other things, as an intermediate While our goal is to create a unified methodology for the and final vehicle for functional validation of the system in pre- and post-silicon domains, the major differences between what is known as post-silicon validation. the platforms dictate differences in the respective implemen- Post-silicon validation is not a new idea and has been used tation of the methodology. One difference is that post-silicon for many years in many places. It can be credited with the platforms provide significantly higher execution speeds, which findings of many functional bugs that escaped pre-silicon ver- has implications on the verification tools that need to be ification. However, in general, functional verification method- adjusted for the best utilization of the available speed (see ology for pre-silicon is still more varied and mature than for section IV). Another difference between the platforms is that post-silicon platforms. Very little is published on post-silicon common pre-silicon simulation platforms, including software verification methodologies (e.g., [2]), and most research in simulators and hardware acceleration platforms (which per- post-silicon validation has centered on on-line checking and form the simulation on dedicated hardware and therefore offer debugging capabilities of the silicon platforms (e.g., [3]–[7]). much higher simulation speeds than software simulators) sup- In recent years, we have seen more evidence that pre- and port a detailed level of observability into the state of the design post-silicon verification cannot achieve their goals on their as it is being tested [11], [12]. The silicon provides fewer own; pre-silicon, in terms of finding all the bugs before tape- opportunities to observe the behavior of the system. Therefore, out, and post-silicon, in terms of finding the bugs that escaped bridging the gap between the pre- and post-silicon verification domains cannot be done blindly by copying methodologies 978-3-9810801-7-9/DATE11/ c©2011 EDAA and technologies from one domain to the other.

        In fact, the limited observability of the silicon raises the    Test Program Template           Test Program
need to use accelerators as a third platform in the proposed            Variable: addr = 0x100          Resource Initial Values:
                                                                        Variable: reg                    R6=8, R3=-25,.. R17=-16
methodology, in addition to simulation and the silicon itself.          Bias: register-dependency        100=7, 110=25,.. 1F0=16
Accelerators are fast enough to run simple post-silicon tools,          Instruction: Store R5 → ?       Instructions:
and are observable enough to allow coverage measurement.                Repeat (addr < 0x200)            500:  Store R5 →     FF0
                                                                                                         504:  Load  R4 ← 100
This allows us to measure coverage of the post-silicon tools             Instruction: Load reg ← addr    508:  Sub  R5 ← R6-R4
in a silicon-like environment.                                           Select                          50C:  Load  R4 ← 110
                                                                          Instruction: Add ? ← reg + ?   510:  Add  R6 ← R4+R3
          The focus of the paper is the stimuli generation aspect of      Bias: sum-zero                 :
the unified methodology. We show how many of the concepts                 Instruction: Sub ? ← ? - ?     57C:  Load  R4 ← 1F0
and technologies that made constrained random test generators            addr = addr + 0x10              580:  Add R9 ← R4+R17
successful in pre-silicon verification can be adapted to the                  Fig. 1.     test-template and corresponding test
post-silicon domain. Specifically,    we discuss         how a user-
directable tool that uses declarative test-templates can be used
in post-silicon validation. The differences between the pre- and              II. PRE-SILICON STIMULI GENERATION
post-silicon platforms mean that we cannot simply convert a               A pre-silicon stimuli generator has to provide the user with

successful pre-silicon generator into a post-silicon tool. For the ability to specify the desired scenarios in some convenient example, to avoid spending too much time on generation way, and produce many valid high-quality test cases according compared to execution of the tests, the heavy sophisticated to the user’s specification. These scenario specifications - generation engine used in pre-silicon generators needs to be known as test-templates - are written in a language that should replaced with a lighter and much faster engine. In addition, to enable an easy and accurate way for specifying the scenarios avoid long loading time, the post-silicon solution needs to be from the verification plan. Figure 1 shows an example of a test- in the form of an exerciser, that is, a tool that continuously template that defines a table-walk scenario (on the left) and an generates test cases, executes them, and check their results. example of a test generated from this template (on the right). We implemented the proposed methodology in a tool called The test-template is written in the test-template language of Threadmill [13]. Threadmill is a bare-metal exerciser, i.e., GenesysPro- [14] - IBM’s well-established test generation tool an exerciser that operates on the silicon itself without the for the functional verification of processors using a software support of an operating system. Threadmill fulfills the above simulator platform. The rest of this section describes Genesys- requirements; it is a directable random generator with a simple Pro’s approach to test generation. and fast generation engine. In addition, it meets other require- The scenario of figure 1 starts with a Store and then a ments of exercisers, such as providing mechanisms to assist in sequence of Loads, each followed by either an Add or a checking. Threadmill also addresses technical challenges, such Sub instruction. The memory locations accessed by the Load as how to replace the reference model, which is an essential instructions are contiguous in memory as seen in the Resource part of both generation and checking in pre-silicon. Initial Values section of the test (addresses 0x100–0x1F0). The proposed unified methodology and Threadmill were This is managed by a test-template variable addr. used for the first time in the verification and bring-up of the The use of test-templates thus separates the test planning ac- IBM POWER7 processor with encouraging results. The use tivity from the generator’s development activity. The language of Threadmill (and other exercisers) on accelerators provided consists of four types of statements: basic instruction state- significant help to the pre-silicon verification effort in finding ments, sequencing-control statements, standard programming many bugs, some of them severe. The coverage feedback constructs, and constraint statements. Users combine these at this stage provided a means for building a good set of statements to describe complex tests that capture the essence test-templates for the bring-up team, which helped ensure a of the targeted scenarios, leaving out unnecessary details. smooth bring-up and reduced the number of escapes from The generated test cases must be valid according to the the first tape-out [11]. In addition, the user’s ability to direct processor’s architecture, and satisfy the user’s request specified Threadmill through test-templates aided in recreating some in the template. In addition, they should also be different from bugs found on silicon in a pre-silicon environment, which each other as much as possible. This is done by specifying helped analyze and fix these bugs. the rules determining the validity of a test case, as well as the The rest of the paper is organized as follows: in Section II, user’s requests as constraints. The generator then produces a we describe the main properties of a pre-silicon stimuli gener- test case by random sampling of the solution space to the ator, as a reference. In Section III, we define the requirements resulting constraint satisfaction problem [15]. for a directable post-silicon stimuli generator. Section IV The distribution of the generated tests should not be uni- describes the proposed unified methodology. Section V de- form, as we want to favor tests that include interesting scribes Threadmill, our post-silicon directable exerciser. Sec- verification events (e.g., register dependency, memory colli- tion VI shows our experience with the unified methodology sions), especially ones that are extremely unlikely to occur and Threadmill in the verification of the POWER7 processor. under uniform distribution. This is done by having knowl- Section VII concludes the paper. edge embedded in the generator, allowing it to bias random

 Post-Silicon                    Ee                                      The difference in speed between pre- and post-silicon plat-
 Threadmill Exerciser Flow                                             forms affects the way the platform cycles are utilized. While
                                                                     pre-silicon test-generators strive to use the scarce simulation
 ETE                                                     =              cycles as effectively as possible by using extensive testing
 ]                 i             =                                   knowledge to generate very high-quality test-cases, the silicon
 |                 |         [|                          =              platform is anything but short on run-time cycles. Therefore
                                                                       to divert the resources from generation to execution, a post-
      Model ofthe      !     I=]*\      |                           silicon exerciser should spend less effort in generating precise
 LEN                                                                   interesting scenarios and instead generate more general test-
 FO                                                                   cases using a lightweight generator. This loss of precision is
 Pre-Silicon           Test                           = ==           compensated by an increase in number of tests generated.
 Genesys-Pro Fon”      —         -                                      Another area where the pre-silicon platforms differ from the
                                                                     silicon platform is the level of observability. This has a deep
                                                                      impact both on the ability to perform checking and the ability
          Fig. 2.  Threadmill Vs. Genesys-Pro tool flows                to measure coverage. Pre-silicon checking techniques include
    for example scoreboards, and assertions [1] which depend on
    a high level of observability into the design which is not

decisions towards stimuli that cause interesting events [15]. available in typical silicon platforms. Similarly, the ability to This testing knowledge defines the interesting verifications measure coverage on the silicon is very limited due to the low events, including the stimuli that trigger them. As the stimuli observability. We overcome this problem by running our post- for some interesting events depend on the processor’s state, silicon exerciser on the acceleration platform where coverage the generator also employs a reference model of the DUV, can be measured as detailed in Section IV. simulating on it every generated instruction. This way the Another requirement for a post-silicon exerciser is sim- generator maintains an accurate view of all the architectural plicity. Hardware failures are hard to debug and therefore resources, and takes them into account during the generation simple software must be used to ease the effort. In addition, of interesting events. This scheme is shown in the lower part we’d like to deploy the tool during very early stages of the of Figure 2. post-silicon validation effort when OS can’t be run on the Genesys-Pro has been in use by IBM for over ten years. DUV and operations such as reading files from an I/O device It has proven to be effective in meeting users’ requirements, are not supported. Another requirement is to keep the ratio enabling them to write test-templates implementing the core between time spent in generating and checking the test-case verification plans of IBM’s complex processors [16]. and the time spent running it as low as possible. For example, while embedding a reference model in an exerciser could III. POST-SILICON STIMULI GENERATION improve its checking capability, the reference model itself is complicated software that would significantly reduce the The characteristics of the post-silicon platform and the dif- platform’s utilization. Examples of checking techniques that ferences between it and pre-silicon software simulators dictate don’t use a reference model are mentioned in Section V. differences in the best ways to use this platform for functional verification in general, and to stimuli generation specifically. IV. A UNIFIED VERIFICATION METHODOLOGY The first important characteristic is the long loading and To better integrate post-silicon validation to the overall initialization time of the silicon. Consider, for example, a verification process and improve its synergy with pre-silicon test-case of 10,000 instructions. Such a test-case takes less verification, we need a unified verification methodology that is than one millisecond to complete on an advanced processor. fed from the same verification plan source. A key ingredient On the other hand, preparing the processor for the execution for the success of such methodology is providing common (i.e., performing a power-on sequence, loading the test-case to languages for the pre- and post-silicon aspects of it in terms of memory, etc.) can take more than a minute. This results in a test specification, progress measure, etc. Figure 3 depicts such execution utilization of less than 0.002%. a methodology. This verification methodology leverages three As a result, post-silicon validation tends to rely more on different platforms: simulation, acceleration, and silicon. The longer running solutions. One commonly used solution of this methodology requires three major components: a verification type is the use of available applications, including operating plan, directable stimuli generators suited to each platform, and systems. Another solution is the use of exercisers, which are functional coverage models. Note that important aspects in programs that run on the DUV and “exercise” it by testing any verification methodology, such as checking, are omitted interesting scenarios. An exerciser is a self-contained solution. from the figure to maintain focus on the main aspect of our It generates the test-cases, runs them, does the checking, and contribution, namely stimuli generation. contains OS services required by the test-cases. The exerciser The verification plan includes a long list of line-items, each runs in an endless loop which makes it a good post-silicon targeting a feature in the DUV that needs to be verified. solution since it is only loaded once on the DUV. Each such feature is associated with coverage events that the

                Functional Verification  Plan |                        also dictate differences in the way test-templates are written
                                                                       for pre- and post-silicon tools. A test-template could be very
               Pre-Silic
                                                                       be more general leaving more room for randomization. The
            fest      ==     Post              Silicon                 specific and describe a small set of targeted tests or it could
Coverage                                                               validation engineer writing test-templates for a post-silicon
                ET                                                     exerciser must bear in mind the fact that the test-template is
            on    Simulation  on      Accelerator                      used to generate a huge number of test-cases and get many
                                                                       processor cycles. To effectively use these test cycles, the test-
            Fig. 3. A unified verification methodology                 template must allow for enough interesting variation. A test-
                                                                       template that is too specific will quickly “run out of steam”
                                                                       on silicon and start repeating similar tests. A pre-silicon test-
verification team expects to observe during the verification           template on the other hand would typically be more directed
process and the methods to be used to verify the feature. The          to ensure that the targeted scnearios are reached within the
verification plan is implemented using random stimuli gener-           fewer cycles available on simulation.
ators that produce a large number of test-cases, and coverage
tools that look for the occurrence of events in the verification           V. THREADMILL
plan. The random stimuli generators are directed towards the                      Threadmill was developed to enable the unified method-

verification goals by using test-templates. The test-templates ology described in Section IV, i.e., to support a verification allow the generators to focus on areas in the DUV ranging process guided by a verification plan by enabling validation from large generic areas, like the floating-point unit, to very engineers to guide the exerciser through test-templates. The specific areas, like a bypass between stages of the pipeline. high-level tool architecture of Threadmill is depicted in Fig- Coverage analysis identifies gaps in the implementation of the ure 2, along with the flow of Genesys-Pro [9] - the pre-silicon plan. Its feedback is used to modify test-templates that do not test generator tool described in Section II. fulfill their goals, and create new ones. Like Genesys-Pro, the main input to Threadmill is a test- Extending this methodology to post-silicon validation is template that specifies the desired scenarios. For reasons difficult because the limited observability of the silicon does mentioned in Section IV, the templates used for pre- and post- not allow to measure coverage on the silicon. To overcome silicon tests have different characteristics. The test-template this problem, we leverage the acceleration platform to measure language of Threadmill is very similar to the language of coverage of post-silicon tools. To take advantage of the Genesys-Pro. However, to adhere to the simplicity and gen- coverage information collected by the accelerators and use it eration speed requirements, several constructs that require in the post-silicon, shortly before first silicon samples come long generation time, such as events, are not included in back from the fab, a regression suite of exerciser test-templates Threadmill’s language. Other inputs to Threadmill include the is created based on the coverage achieved on the accelerators. architectural model and testing knowledge and the system This regression suite is then used to continue the verification topology. Again, for simplicity reasons, many testing knowl- process on the silicon platform. edge items that are included in Genesys-Pro models are not With the unified methodology, we attach to each of the line- used by Threadmill. items in the verification plan one or more target platforms on The Threadmill execution process starts with a builder which it will be verified. These line-items are converted to application that runs off-line to create an executable exerciser test-templates in the languages of the generation tools used by image. The role of the builder is to convert the data incorpo- each platform. A key ingredient for the success of the unified rated in the test-template and the architectural model into data methodology is similar operation of the stimuli generators. structures that are then embedded into the exerciser image. In this sense, we would like the generators to use the same This scheme eliminates the need to access files or databases test-template language, and when provided with the same test- while the exerciser is running. template, we would like the tools to produce similar (though The exerciser image is composed of three major compo- not identical) test-cases. Of course, the different platforms pro- nents: a thin, OS-like layer of basic services required for vide different opportunities and put different constraints and Threadmill’s bare-metal execution; a representation of the requirements on the generation tools, but whenever possible, test-template, architectural model, and system configuration there are advantages to having similar tools. First, the pre- and description as simple data structures; and fixed (test-template post-silicon teams can share the task of understanding the line- independent) code that is responsible for the exercising. The items in the verification plan and planning ways to test them. executable image created by the builder is then loaded onto In addition, the common language allows for easier adaptation the silicon platform where the exerciser indefinitely repeats of test-templates from one platform to another. For example, the process of generating a random test case based on the when a bug is detected on the silicon platform, narrowing test-template, the configuration, and the architectural model, down the test-template and hitting it on the simulation platform executing it, and checking its results. eases the root-cause analysis effort. In the case of Genesys-Pro, the test generation process It’s important to note that the differences between platforms is carried outside of the simulation environment (say on a

dedicated server) and only the generated tests are loaded and project with the goal of revealing as many defects as possible. run on the simulation platform. Simulation cycles would be As soon as the core was sufficiently stable, the Exercisers too slow to allow generation during simulation. The “off-line” on Accelerators (EoA) phase started and was executed in generation, on the other hand, can afford to spend time on parallel to simulation. Following every tape out, a post-silicon sophisticated generation and checking, for example, by using validation phase started. This process was accompanied by a reference model as seen in Figure 2 for Genesys-Pro. Thread- a supporting effort on both the simulation and acceleration mill’s test generation component was designed to be simple platforms to assist the analysis of bugs that were initially and fast. The generation is therefore static, i.e., without the use detected on the silicon and the verification of the bug fixes. of a reference model. Reference models provide the generator The influence of the unified methodology on the POWER7 information about the state of the processor before and after verification started in the EoA phase. Exercisers and acceler- the generation of each instruction. This information is used for ators have been used in previous projects [16], but POWER7 checking but also to create more interesting events. Reloading was the first time in which the EoA effort was tightly inte- resources, such as registers, can be a partial replacement to the grated in the overall verification effort of the core. Each ex- reference model, but this solution potentially interferes with erciser was assigned several line items in the core verification the generation of the requested scenarios. For data-oriented plan and was asked to provide test cases that cover the events events, such as divide-by-zero, a simple yet effective solution associated with these line items. The coverage measured on the is to reserve registers to hold interesting values. Of course, accelerators was incorporated in the project coverage reports the generator has to ensure that the reserved registers are not together with unit and core simulation coverage. Accelerator modified during the test. coverage was also used to harvest high-quality test-templates Execution of the same test case multiple times is used for post-silicon validation. as a partial replacement for checking done by the reference Threadmill was not the only exerciser used in POWER7. model. This is done by comparing certain resource values, Several other exercisers were also used, and each of the such as registers and part of the memory, for consistency in exercisers was assigned its own items in the verification plan, different executions of the test case. Running the same test based on its capabilities. Each exerciser created test cases for case multiple times may result in different results even when the verification items in its own way. Some exercisers used bugs are not present. For example, when several threads write modification of their base software; others used parameters, to the same memory location, the final value at this location tables, and macros for that purpose. Threadmill was the only depends on the order of the write operations. This requires exerciser that used declarative test-templates. This provided that certain mechanisms be implemented in the generator to Threadmill with several advantages. First, it provided a con- restrict the number of unpredictable resources. Although the venient way for tool users to specify the desired scenarios. It multi-pass comparison checking technique is limited, it has also allowed easy sharing of knowledge between Threadmill proven to be effective when control-path-oriented bugs, or and Genesys-Pro due to the use of a similar language. Finally, bugs that reside in the intersection of the control- and data- it provided a means for easier transfer of bug information paths, are concerned. To increase the probability of exposing between the platforms for bug recreation and fix validation. such bugs, it is beneficial to introduce some kind of variability The use of EoA provided several advantages to POWER7. into the different execution passes, while making sure that First, the EoA process caught some high impact bugs. One the variability maintains the predictability of the compared such bug detected by Threadmill shortly before the first resources. This can be done, for example, by changing the tape out was a show-stopper bug that would have totally machine mode, or changing thread priorities. undermined the first post-silicon validation process and would VI. POWER7 EXPERIENCE have required an additional tape out. In terms of coverage, the EoA effort obtained results that are comparable with the core The proposed unified methodology and Threadmill were simulation effort. Table I shows the coverage results shortly used in the verification of IBM’s POWER7 processor. The before the first tape out for unit simulation, core simulation, POWER7 processor implements the 64-bit IBM Power Archi- and EoA. The table shows that for control-oriented units, tecture. Each POWER7 chip incorporates eight SMT processor such as the fetch unit (IFU) and sequencing unit (ISU), EoA cores with three levels of caches, memory and I/O controllers, coverage is almost similar to the core simulation coverage. and other support and management logic. The processor cores This is an indication of the exercisers’ ability to reach inter- are out-of-order superscalar cores supporting up to four simul- esting scenarios even in a small number of acceleration cycles taneous threads. Each core contains twelve execution units (compared to the silicon platform). that are shared between the threads. This section highlights The shared verification plan and the EoA effort gave a boost some of the results and lessons learned from this first use of to the post-silicon validation phase. The organized verification the unified methodology and Threadmill in a large industrial plan and the harvested test-templates provided the bring-up project. It focuses on the core verification of the POWER7. team with a good starting point to their effort and a means POWER7 was verified using three platforms: simulation, to track progress. The heavy use of the exercisers in EoA acceleration, and silicon. Traditional pre-silicon simulation- also helped improve their readiness for the bring-up. All based verification was used throughout the lifetime of the this supported a much shorter bring-up phase. In addition,

 DUV Unit   Unit Sim  Core Sim             EoA     Total         test-templates. Other aspects, such as the trade-off between
 IFU         96.79     96.77               94.99   98.65         generation smartness and generation speed, were adapted to
 ISU         96.48     92.49               92.78   97.42
 FXU         99.60     84.72               85.85   99.85         better fit the requirements of the post-silicon domain.
 FPU         97.44     98.15               90.20   99.58                            The resulting tool, a bare-metal exerciser called Threadmill,
 LSU         94.33     91.04               85.32   98.66         was   used   in    the    pre-  and       post-silicon  verification of      the
 PC          92.51     76.95               55.23   93.51         POWER7 processor chip. Results of this experience confirm
 Core Total  96.18     92.78               88.70   98.06         our beliefs about the benefits of the increased synergy between
     TABLE I                                                     the pre- and post-silicon domains and of using a directable
     POWER7 COVERAGE RESULTS                                     generator in post-silicon validation.
                                                                                             We see several ways to enhance the work described in
                                                                 this paper.     First, we would like to                 extend this    “bridging
                                                                 the gap” concept to other aspects of functional verification,
compared to previous projects, a larger percentage of complex    such as checking and coverage. In addition, we are working
 bugs were found earlier in the bring-up process and by bare-    to incorporate more testing knowledge into Threadmill. For
metal tools, which are easier to debug.                          example, we are working to improve Threadmill’s ability to
      The unified methodology and the similar test definition    create interesting address translation paths. This will enable
      languages of Genesys-Pro and Threadmill provide another    Threadmill to better exercise areas such as cache coherency

advantage in the ability to hasten root-cause analysis of bugs and the memory and I/O subsystems. and validation of bug fixes. This process produced a synergy between platforms where each task in the recreation, analysis, REFERENCES and fix validation of the bug was performed on the most suit- [1] B. Wile, J. C. Goss, and W. Roesner, Comprehensive Functional able platform. For example, the relative speed of accelerators [2] Verification - The Complete Industry Cycle. Elsevier, 2005. compared to simulators was used to look around a complex H. G. Rotithor, “Postsilicon validation methodology for microproces- sors,” IEEE Design & Test of Computers, vol. 17, no. 4, pp. 77–88, bug found by Genesys-Pro in simulation. This was done by 2000. generalizing and adapting the test-template that produced the [3] M. Abramovici, P. Bradley, K. Dwarakanath, P. Levin, G. Memmi, and bug to Threadmill. Another example started with a bug that D. Miller, “A reconfigurable design-for-debug infrastructure for socs,” in Proceedings of the 43rd Design Automation Conference, July 2006, was found on silicon. The test case that found the bug (that pp. 7–12. was not found by Threadmill) and basic observations from the [4] K.-h. Chang, I. L. Markov, and V. Bertacco, “Automating post-silicon silicon caused the designer to speculate on a reason for the bug. debugging and repair,” in Proceedings of the 2007 international confer- ence on Computer-aided design, November 2007, pp. 91–98. The next step was to create test-templates for Threadmill (on [5] F. M. De Paula, M. Gort, A. J. Hu, S. J. E. Wilton, and J. Yang, the accelerator) to test that theory. Once the bug was recreated “Backspace: formal analysis for post-silicon debug,” in Proceedings of on the accelerator, the test-template was narrowed down and the 2008 International Conference on Formal Methods in Computer- Aided Design, November 2008, pp. 1–10. the bug was recreated in simulation. Finally, after the bug was [6] I. Wagner and V. Bertacco, “Reversi: Post-silicon validation system for fixed, all the test-templates created for its recreation were used [7] modern microprocessors,” in ICCD, 2008, pp. 307–314. on the appropriate platforms to ensure that the fix was correct. K. Chen, S. Malik, and P. Patra, “Runtime validation of memory ordering using constraint graph checking,” in HPCA, 2008, pp. 415–426. The success of the unified methodology was evident in the [8] H. B. Carter and S. G. Hemmady, Metric Driven Design Verification: bug shift from silicon to acceleration and in the productivity An Engineer’s and Executive’s Guide to First Pass Success. Springer, improvements that it gave to the post-silicon, acceleration, and [9] 2007. M. L. Behm, J. M. Ludden, Y. Lichtenstein, M. Rimon, and M. Vinov, pre-silicon teams. The verification of POWER7 is considered “Industrial experience with test generation languages for processor by experts to be the best executed in IBM in recent years. [10] verification,” in DAC, 2004, pp. 36–40. The low number of bugs that escaped to silicon and escaped A. Piziali, Functional Verification Coverage Measurement and Analysis. Springer, 2004. from the first to the second tape out provides good supporting [11] A. Adir, A. Nahir, A. Ziv, C. Meissner, and J. Schumann, “Reaching evidence of this. We believe that the wour work has made a coverage closure in post-silicon validation,” in Proceedings of the 6th significant contribution to POWER7’s success. [12] Haifa Verification Conference, 2010. J. Darringer et al., “EDA in IBM: past, present, and future,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and VII. CONCLUSIONS [13] Systems, vol. 19, no. 12, pp. 1476–1497, December 2000. The growing importance of post-silicon validation to the Building a bridge: from pre-silicon verification to post- silicon validation. [Online]. Available: http://es.fbk.eu/events/fmcad08/ verification process increases the need for synergy between presentations/tutorial moshe levinger.pdf the pre- and post-silicon verification methodologies and tech- [14] A. Adir, E. Almog, L. Fournier, E. Marcus, M. Rimon, M. Vinov, nologies. This paper presented a unified verification method- and A. Ziv, “Genesys-Pro: Innovations in test program generation for functional processor verification,” IEEE Design and Test of Computers, ology based on a common verification plan and coverage vol. 21, no. 2, pp. 84–93, 2004. models and that uses similar test-template languages. To [15] Y. Naveh, M. Rimon, I. Jaeger, Y. Katz, M. Vinov, E. Marcus, and implement the methodology, we transferred and adapted the G. Shurek, “Constraint-based random stimuli generation for hardware verification,” in AAAI, 2006. pre-silicon stimuli generation methodology into post-silicon. [16] D. W. Victor et al., “Functional verification of the POWER5 micropro- In this transition, we maintained the concepts of directed cessor and POWER5 multiprocessor systems,” IBM Journal of Research random stimuli generator that is controlled via declarative and Development, vol. 49, no. 4, pp. 541–554, 2005.