About This Book 1
The MindShare Architecture Series 1
Cautionary Note 2
This Book Assumes PCI Background Knowledge 3
Specifications This Book Is Based On 3
Organization of This Book 3
Who This Book Is For 4
Prerequisite Knowledge 4
Documentation Conventions 5
Hexadecimal Notation 5
Binary Notation 5
Decimal Notation 5
Bits Versus Bytes Notation 5
Bit Fields(Logical Groups of Bits or Signals) 6
Timing Diagram Drawing Convention 6
Clock-by-Clock Timing Diagram Description 8
Signal Polarity 8
Visit Our Web Site 8
We Want Your Feedback 9
Part 1：Basic Concepts 13
Chapter 1：PCI Needed Improvement 13
Wait States Yield Poor Performance 13
Relatively Slow Clock Speed 14
Transfer Size Unknown 14
PCI Delayed Transactions Are Inefficient 15
Initiator Retries Use Up Valuable Bus Time 15
Initiator Doesn′t Supply Transfer Count 15
Delayed Completion 16
Initiator′s Transaction ID Is Sketchy at Best 16
Snoops Hurt Performance 16
Host/PCIX Bridge Knows AGP′s Area of Memory Is Non-Cacheable 16
Memory Used by PCI Masters May or May Not Be Cached 17
Snoops Slow Down PCI Accesses to Main Memory 17
Snoop Traffic on Processor Bus Can Hurt Processor(s) 17
Main Memory Less Available to Processor(s) 17
Latency Timer Use Not Optimized in PCI 19
Min_Gnt Register：Timeslice Request 19
Latency Timer：“Timeslice”Register 19
Insufficient Info to Select a Good Value 20
Data Phase Parity Error Recovery Usually Not Possible 21
PCI Data Parity Error Recovery 21
Important Note Regarding Chipsets That Monitor PERR 22
No Indication of Device Width 22
MSI Feature Optional in PCI Environment 22
Introduction 22
Advantages of MSI Interrupts 23
Power Management Optional 23
Legacy PCI Devices—No Standard PM Method 23
Device Support for PCI PM Optional 23
Discovering Function′s PM Capability 23
Configuration Software Constrained by 32-Bit Memory BARs 26
Stepping Yields Poor Performance 27
Chapter 2：PCI-X Improves on PCI 29
PCI-X Is Backward-Compatible With PCI 29
PCI-X Is More System-Centric Than PCI 30
Higher Clock Speeds Possible 31
Wait States Eliminated 31
Data Transferred in Blocks 31
In PCI，Data Transferred as a Series of Data Items 31
In PCI-X，Data Transferred as a Series of Blocks 32
Master Cannot Delay Transfer of First Data Block 32
Target Can Only Delay Transfer of First Block 32
Master and Target Can Only Disconnect on Block Boundaries 32
But…There Are Two Exceptions 32
Disconnecting on Block Boundaries 33
Master Disconnection of a Transfer 33
Target Disconnection of a Transfer 33
Latency Timer Usage 33
Requester and Completer 34
Transfer Size Specified 34
Requester ID and Transaction ID Are Specified 34
Split Transactions Replace Delayed Transactions 35
Target Cannot Transfer Data Within 16 Clocks From FRAME# Assertion 35
Long-Latency Access Handling in PCI 35
Long-Latency Access Handling in PCI-X 35
Split Transactions More Efficient Than Delayed 36
Example：Requester and Completer on Same Bus 37
Example：Requester and Completer on Different Buses 38
Dynamic Traffic Analysis and Load Tuning 40
Data Phase Parity Error Recovery 41
General 41
Recovery Usually Not Possible in PCI 41
PCI-X Chipset Does Not Monitor PERR 42
What the PCI-X Spec Says 42
Requester Hardware Doesn′t Handle Error Itself 42
With Appropriate Master and Device Driver Design，Recovery Possible 42
Device Driver With No Recovery Capability=SERR 43
If Bus in PCI Mode，Bridge Monitors PERR 43
64-Bit Connection Indication 44
MSI Feature Mandatory 44
Power Management Mandatory for Add-In Devices 44
Snoops Can Be Eliminated 44
Memory BARS Must Be 64-Bit Width 45
Bus Masters That Access Memory Must Support DAC Command 45
Stepping Eliminated 45
Fast Back-to-Back Transactions Eliminated 46
Chapter 3：Lowest Common Denominator Defines Mode 48
Bus Protocol/Speed=Lowest Common Denominator 48
Discovering a PCI-X Bus 48
Discovering PCI Devices on a PCI-X Bus 49
Some Example Systems 49
System With No Connectors on PCI-X Bus 49
Single Bus System With Connectors 49
Systems Supporting Both PCI-X and PCI Environments 52
Introduction 52
System With Single Host/PCIX Bridge 52
Dual Host/PCIX Bridge System 54
Chapter 4：Device Types and Bus Initialization 56
All Devices Support 33MHz PCI 56
General 56
133MHz PCI-X Device Must Support 33MHz and 66MHz PCI 56
When M66EN Is Grounded on a Card 56
Effect of Grounded M66EN on a PCI Source Bridge 56
Effect of Grounded M66EN on a PCI-X Source Bridge 56
M66EN Usage on a 66MHz PCI Card or a PCI-X Card 57
All PCI-X Devices Support 66MHz PCI-X Mode 57
PCIXCAP Indicates Protocol/Frequency Required 58
General 58
PCIXCAP Indicates Capabilities of Card′s First Device 61
Device Not Permitted to Use PCIXCAP as Input or Output 61
Bridge′s Interpretation of M66EN and PCIXCAP 61
Maximum Reliable Speed Verified by Design and Testing 63
Supplying PCI-X Devices With Protocol and Speed 64
General 64
Host/PCIX Bridge Pattern Delivery Sequence 64
PCIX-to-PCIX Bridge Pattern Delivery Sequence 65
The Sequence 65
But First 65
Why Bus Must Be Idle When Init Pattern Driven 67
Upon Receipt of Pattern，Device Initializes Itself 67
Init Pattern Setup and Hold Time 67
General 67
Relationship With Trhff 68
Reassertion of RST# Necessitates Redelivery of Pattern 68
General 68
Secondary Bus RST# Follows RST# on Primary Side 68
Secondary Bus RST# Under Software Control 69
Bridge Must Support Interfaces in Different Modes 70
Hot-Plug PCI-X Bus Initialization 71
Some Systems Permit Hot-Plug of PCI-X Cards 71
For Background on PCI Hot-Plug 71
Problems Associated With PCI-X Hot-Plug 71
Determination of Card Capabilities 72
General 72
Determining the Presence of a PCI Card 73
Unacceptable Method 73
Acceptable Method 73
Determining the Presence of a 66MHz PCI Card 74
Determining the Presence of a 66MHz-Capable PCI-X Card 74
Determining the Presence of a 133MHz-Capable PCI-X Card 75
Hot-Install of Card With PCIX-to-PCIX Bridge on It 75
Description 75
How to Avoid Bus Renumbering During Run-Time 76
Hot-Install of Card With PCI-to-PCI Bridge on It 77
Checking PCIXCAP Without Power Applied to Card 78
Changing Secondary Bus Number Configuration Register 78
Conservation of Address Space 78
General 78
Requesting Memory Space 79
Requesting IO Space 80
Early Configuration Access to Newly-Installed Device 80
Possible Adjustment of Max Memory Read Byte Count Values 80
Placing Device in Low-Power Mode Requires Quiesce 81
Chapter 5：PCI-X Is a Registered Bus 83
PCI-X Is a Low-Voltage Swing(LVS)Bus 83
Introduction to the Registered Nature of the Bus 84
Address/Command Decode Example 85
Data Read Example 86
Chapter 6：Intro to Commands 89
Commands Fall Into Three Categories 89
Command Encoding 90
Dword Commands 92
General 92
IO Read and Write Commands 92
Memory Read Dword Command 92
Configuration Read and Write Commands 93
Interrupt Acknowledge Command 93
Special Cycle Command 95
Burst Commands 95
All Burst-Oriented Commands Are Memory Transfers 95
Start Address Is Byte-Aligned 95
Transfer Length 95
Linear Addressing Is Implied 96
Memory Read Block Command 96
Alias To Memory Read Block Command 96
Memory Write Block Command 97
Alias To Memory Write Block Command 97
Memory Write Command 97
Split Completion Command 97
Dual-Address Cycle(DAC)Command 98
Chapter 7：Intro to Transaction Phases 100
The PCI Transaction Phases 100
The PCI-X Transaction Phases 100
Address Phase 100
Attribute Phase 100
Response Phase 101
Data Phase(s) 101
Chapter 8：Intro to Transaction Termination 103
Introduction 103
Initiator Termination of Transaction 104
Byte Count Satisfaction 104
Initiator Approaching Buffer Full or Dry Condition 104
Connection Timeout 105
Early Target Termination of Transaction 106
General 106
Target Abort 107
Reasons for a Target Abort 107
Target Abort Always Ends a Transaction 107
Retry 107
Reasons for Issuing a Retry 107
Initiator′s Response to a Retry 108
Response to a Retry in PCI 108
Response to a Retry in PCI-X 108
Detailed Description of Retry 108
Single Data Phase Disconnect 108
DiSconnect At Next ADB 109
Split Response 109
Definition 109
Commands That Will Not Receive a Split Response 110
Bridge Handling of Memory Writes 110
Bridge Handling of Transactions Other Than Memory Writes 110
Addressing a Location Within the Bridge 111
Addressing a Location on the Opposite Side of Bridge 111
Chapter 9：Intro to Split and Immediate Transactions 114
Definition of Requester and Completer 114
Definition of a Sequence 114
Definition of Requester ID，Tag，and Sequence ID 114
How Does a Device Know Its Requester ID? 114
Immediate Transaction 116
Definition of an Immediate Transaction 116
Immediate Completion Completes the Sequence 116
…Unless It′s a Memory Write or Memory Write Block 116
…Or Unless First Data Phase of Memory Write Receives Retry 117
Memory Writes Are Posted 117
General 117
Can Be Initiated Before All Write Data Is Ready 117
Split Transactions 118
What Problem Do Split Transactions Solve? 118
Split Completion Uses Sequence ID as the Address 118
Requester and Completer May Reside on the Same Bus 119
Example 119
Split Completion May Contain a Message 119
Memory Read May Result in Multiple Split Completion Transactions 120
Completer Has All Data Buffered Up：One Split Completion 120
Some Data Buffered Up：Multiple Split Completions 120
Requester and Completer May Reside on Different Buses 121
Example 121
Case One：Completer Issues a Split Response 122
Case Two：Completer Transfers Data Immediately 122
Requester Buffer Space Management 124
Allocate Buffer to Hold All of the Data，Then Arbitrate 124
On Split Response，Requester Must Commit Buffer Space 125
On Immediate Response，Requester Can Release Buffer Space 125
Bridge Can Retry or Disconnect Split Completion 125
Requesters Must Implement a Split Request Timeout 125
Part 2：Transaction Protocol 129
Chapter 10：Bus Arbitration 129
Stepping Not Permitted 129
Request and Grant Signals Are Registered 129
Example Arbitration 130
Device Design Rules 133
No Fast Back-to-Back 133
REQ# Can Be Asserted or Deasserted at Any Time 134
Issuig REQ# and Then Changing Your Mind 134
After REQ# Is Asserted，Assert FRAME# or Deassert REQ 134
Initiator Can Start Transaction Two Clocks After GNT# Is Asserted 134
GNT# Removal in Clock Prior to FRAME# Too Late to Stop Initiation 135
Single Idle Clock Between Transactions 135
Multiple Idle Clocks Between Transactions 137
Back-to-Back Transactions by Same Initiator 138
Device Containing Multiple Initiators 138
Arbiter Design Rules 138
No GNT# Asserted and REQ# Detected 138
Must Allow opportunity to Start Configuration Transaction 139
When Bus Not Idle，GNT# Can Be Deasserted Sooner 140
If Transaction Not Started，Arbiter May Ignore Master 140
Removing GNT# From One Initiator and Giving It to Another 141
Parking Recommendation 141
Bus Parking 141
Floating Bus Causes Power Drain 141
Bus Parking Prevents Bus Float 141
Rules Associated with Bus Parking 142
HOW the Initiator Deals With Preemption 143
The Basics 143
After Timeslice Exhaustion and Preemption，How Soon Must It Yield? 144
What Is the Recommended LT Value? 144
How Much Data Can Be Transferred During Default Timeslice? 144
Chapter 11：Detailed Command Description 147
Dword Commands 147
General 147
Command Encoding 148
Illegal to Assert REQ64# in Dword Transactions 148
IO Read and Write Commands 149
Basic Description 149
Start Address and Byte Enable Format 149
Start Address Is Byte-Aligned 149
Byte Enable Usage 149
IO Transaction With No Byte Enables Asserted 149
Target Response to an IO Access 151
General 151
Completer Handling of an IO Access 151
When Bridge Acts as Target of IO Access 151
Memory Read Dword Command 152
Start Address and Byte Enable Format 152
Start Address Is Byte-Aligned 152
Byte Enable Usage 152
Transaction With No Byte Enables Asserted 152
Target Response to a Memory Read Dword Access 152
General 152
Completer Handling of a Memory Read Dword Access 152
Bridge Acts as Target of Memory Read Dword Access 153
Configuration Read and Write Commands 153
Interrupt Acknowledge Command 154
General 154
Background 154
Host/PCLX Bridge Handling of Interrupt Acknowledge 155
System Using APIC Bus to Deliver Interrupts To Processors 157
Non-Intel System Implementation 159
Special Cycle Command 159
General 159
Basic Description 159
Address Phase 160
Attribute Phase 160
Data Phase 160
In PCI-X Special Cycle Transaction，No Wait States Permitted 160
No Target Is Permitted to Respond 160
Ending the Transaction 161
The Message Types 162
Devices That May Initiate a Special Cycle Transaction 163
Software-Initiated Special Cycle Transactions 163
Burst Commands 163
All Burst-Oriented Commands Are Memory Transfers 163
Burst Command Encoding 163
Start Address Is Byte Aligned 165
PCI Memory Addressing 165
PCI-X Memory Addressing 165
Linear Addressing Is Implied 166
Transfer Length 166
In PCI，Transfer Length Is Unknown 166
In PCI-X，Byte Transfer Count Defines End Address 167
Memory Read Block Command 167
Basic Description 167
Can Be Purely Speculative 167
Target Response to Memory Read Block Access 167
Must Commit Buffer Space Before Initiating 169
Alias To Memory Read Block Command 169
Memory Write Block Command 169
Basic Description 169
If Disconnected，Must Resume With Same Sequence ID 170
If Requester Cannot Complete Memory Write 170
Technically，a Write Can Be Purely Speculative，But 171
Target Never Permitted to Split a Memory Write 171
Target Response to Memory Write Block Transaction 171
Doesn′t Need All of the Write Data Ready Before Initiating 172
Emulating the PCI Memory Write and Invalidate Command 172
Alias To Memorg Write Block Command 174
Memory Write Command 174
Basic Description 174
Can Be Purely Speculative 175
Target Never Permitted to Split a Memory Write 175
Target Response to Memory Write Transaction 175
Doesn′t Need All of the Write Data Ready Before Initiating 175
Split Completion Command 176
Basic Description 176
At Minimum，Completer Must Handle One Split Transaction 176
Can Be Initiated Even If Bus Master Bit Is Off 176
Split Transactions Never Associated With Memory Writes 177
One or More Split Completions to Fulfill One Request 177
Separate Requests May Be Fulfilled out of Order 177
The Address Phase 178
The Attribute Phase 178
Claiming a Split Completion Transaction 180
Requester and Completer Are on the Same Bus 180
Requester and Completer Are on Different Buses 181
Master Abort or Target Abort on a Split Completion 182
Treatment of Byte Enables 182
Requester Must Accept the Split Completion Data 182
When Bridge Is Target of Completion，Can Retry or Disconnect 182
Initiator of Split Completion Can Disconnect 183
After Disconnect，Pick Up Where You Left Off 183
Split Completion Messages 183
To Disconnect First Split Completion at Imminent ADB 184
Disconnecting First Split Completion Is Problematic 184
The Problem 184
The Solution 185
Bridge Buffer Space Problem Due to Corrupted Split Completion 186
Dual-Address Cycle(DAC)Command 187
Chapter 12：Latency Rules 189
Initiator Latency Rules 189
Don′t Start Transfer If You′re Not Ready 189
No Initiator Wait States Permitted…Ever 190
Behavior When Preempted 191
Target Latency Rules 191
Target Response Time During Initialization Period 191
What′s Going on During Initialization Time? 191
Definition of Initialization Period in PCI 191
Definition of Initialization Period in PCI-X 192
Initialization Period and Hot-Plug 192
Target Can Ignore 16-Clock Rule During ROM Shadowing 192
Target Response Time Limit During Run-Time 192
Response Time Limit When No Data Transferred 192
Response Time Limit When Data Transferred 193
Host/PCIX Bridge Must Obey 16-Clock Rule 193
In PCI 193
In PCI-X 193
Subsequent Data Phase Target Latency Rule 193
Maximum Completion Time 193
In PCI 193
How PCI-X Is Different 194
How Can You Prevent Violation of This Time Limit? 195
Chapter 13：The Address，Attribute and Response Phases 198
All Transactions Begin With Address and Attribute Phases 198
Memory Transaction May Have Two Address Phases 199
Attributes Always Delivered on Lower Half of Bus 200
Address/Attribute Format Depends on Command Type 200
Memory Burst Format 200
Description 200
No Snoop Attribute Bit 201
Background 201
Bridge Knows AGP′s Area of Memory Is Non-Cacheable 201
Memory Used by PCI Masters May or May Not Be Cached 202
Snoops Slow Down PCI Accesses to Main Memory 202
Snoop Traffic on Processor Bus Can Hurt Processor(s) 202
Main Memory Less Available to Processors 202
PCI-X Driver Can Make Requester′s Buffer Uncacheable 202
Only Host/PCIX Bridge Pays Attention to NS Bit 203
When NS Is Set and CPU Has a Lock in Force 203
NS Must Be 0 in Some Transactions 203
Dword Command(other than Config)Format 205
Memory Read Dword and Io Format 205
Special Cycle and Interrupt Acknowledge Format 206
Configuration Command Format 206
Two Types of Configuration Transactions 206
Type 0 Configuration Access 207
TYPe 1 Configuration Access 207
Type 0 Configuration Access Format 207
TYPe 1 Configuration Access Format 208
Split Completion Command Format 209
The Response Phase：Connecting With the Target 210
Chapter 14：Dword Transactions 216
General 216
General Format of Timing Diagram Descriptions 216
IO Read and Memory Read Dword 216
Example One 216
Example Two 220
IO Write 224
Example One 224
Example Two 227
Example Three 230
Configuration Read and Write Transactions 234
Interrupt Acknowledge Command 234
Special Cycle Command 235
Chapter 15：Burst Transactions 240
Introduction 240
Short Transfer Within a Block 240
Long Transfer，But Disconnect on First Block Boundary 240
General Format of Timing Diagram Descriptions 240
Memory Read Block Transaction 241
Memory Read Block：Detailed Example 241
Memory Read Block：Variation One 245
Memory Read Block：Variation Two 246
Memory Read Block：Variation Three 247
Memory Read Block：Variation Four 248
Memory Read Block：Variation Five 249
Memory Read Block：Variation Six 250
Memory Read Block：Variation Seven 251
Memory Read Block：Variation Eight 252
Memory Write Block Transaction 253
Memory Write Block：Detailed Example 253
On Burst Writes，Insert Wait States in Pairs 257
Memory Write Transaction 260
Memory Write：Detailed Example 261
Memory Write：Variation One 265
Memory Write：Variation Two 266
Memory Write：Variation Three 267
Memory Write：Variation Four 268
Memory Write：Variation Five 269
Memory Write：Variation Six 270
Memory Write：Variation Seven 271
Memory Write：Variation Eight 272
Memory Write：Variation Nine 273
Split Completion Transaction 274
Split Completion Returning Block of Read Data 275
Returning a Split Completion Error Message 276
Alias To Memory Block Commands 277
Chapter 16：Transaction Terminations 279
General Format of Timing Diagram Descriptions 279
Termination by the Initiator 280
General 280
Byte Count Satisfaction 280
Introduction 280
Ending Transaction of Four or More Data Phases 281
Ending Transaction of Less Than Four Data Phases 282
Introduction 282
Three-Data-Phase Transaction 282
Two-Data-Phase Transaction 284
One-Data-Phase Transaction 286
Initiator Issues Disconnect at Next ADB 289
Initiator Termination Due to Connection Timeout 290
Termination By the Target 290
Target Abort 290
Target Abort Is Always Fatal 290
Some Reasons Target Issues Target Abort 290
Broken Target 290
IO Addressing Error 290
Address Phase Parity Error 290
Initiator′s and Target′s Response to Target Abort 290
Example Target Abort Issued in First Data Phase 291
Example Target Abort Issued in Subsequent Data Phase 293
Target Issues a Retry 294
Target Issues Single Data Phase Disconnect 297
General 297
Special Case Scenario 299
Target Issues Disconnect At Next ADB 299
Disconnect Issued Four Or More Data Phases From ADB 300
Target Issues Disconnect Too Close to Block Boundary 301
To Disconnect When Start Address Very Close to ADB 302
Start Address Three Data Phases From Block Boundary 303
Start Address Two Data Phases From Block Boundary 304
Start Address One Data Phase From Block Boundary 306
Target Issues a Split Response 308
Split Response for a Read 308
Split Response for an IO or Configuration Write 310
Chapter 17：Split Completion Messages 313
Purpose of Split Completion Messages 313
SCM Always Terminates a Sequence 315
Upon Receipt of Error Message，Set Status Bit 315
Message Format 316
Address Phase Format 316
Attribute Phase Format 316
Data Phase Message Format 317
Write Completion Indication 322
Good Completion of Split IO or Configuration Write 322
Bad Completion of a Split IO or Configuration Write 323
Read Completion Indication 323
Good Completion of a Split Dword Read 323
Good Completion of a Split Burst Memory Read 323
Bad Completion of a Split Dword Read 324
Bad Completion of a Split Burst Memory Read 324
General 324
Remaining Byte Count and Remaining Lower Address Fields 324
Device-Specific Error Handling 325
Chapter 18：64-Bit Transactions 327
General Format of Timing Diagram Descriptions 327
64-Bit Data Transfers and 64-Bit Addressing：Separate Capabilities 328
64-Bit Extension Signals 328
REQ64# and ACK64# Have Same Timing as FRAME# and DEVSEL 329
In Attribute Phase，Upper Bus Reserved and Driven High 330
Block Length Remains the Same 330
Bursts Cannot Cross 264 Boundary 330
REQ64# Not Permitted in Dword Transactions 330
General 330
…Unless It′s a Split Completion 331
MSI Write Always Writes a Single 32-Bit Data Value 331
Bridge Must Support DAC on Both Interfaces 331
Width of Function′s Connection to Bus 332
General 332
Add-In Card With a Bridge 334
Determining the Width of a Bridge′s Interfaces 334
64-Bit Cards in 32-Bit Add-In Connectors 335
Pullups Prevent 64-Bit Extension From Floating When Not in Use 336
Problem：A 64-Bit Card in a 32-Bit PCI Connector 336
How 64-Bit Card Determines Type of Slot Its Installed In 338
64-Bit Data Transfer Capability 339
64-Bit Transfers：Only Burst Memory Operations 340
Start Address Byte-Aligned 340
64-Bit Target′s Interpretation of Address 341
32-Bit Target′s Interpretation of Address 341
64-Bit Memory Read Block With 64-Bit Target 341
64-Bit Write Block or Split Completion With 64-Bit Target 347
64-Bit Memory Write With 64-Bit Target 352
Start Address Alignment Defines Data Path Usage 354
Introduction 354
64-Bit to 64-Bit Connection 354
64-Bit to 32-Bit Connection 354
64-Bit Memory Reads From 32-Bit Targets 359
Starting on Even Dword 359
Starting on Odd Dword 365
64-bit Memory Writes to 32-Bit Targets 366
Starting on Even Dword 366
Starting on Odd Dword 368
Example One 368
EXample Two 369
Example Three 370
Example Four 371
Example Five 372
Addressing Memory Above 4GB Boundary 372
Introduction 372
Introduction to the DAC Command 373
DAC Support Mandatory for All Initiators 373
Memory Targets Must Support Wide BARs and DAC 374
Use of DAC Command Changes DEVSEL# and Master Abort Timing 375
Example 32-Bit Transaction Using DAC Command 375
Example 64-Bit Transaction Using DAC Command 377
DEVSEL# Must Not Be Asserted Too Soon 378
Add-In Card Trace Length 379
Parity Generation and Checking 382
Chapter 19：Parity Generation and Checking 383
General Discussion of Parity Generation 383
Parity Generation Is Mandatory 383
In Data Phases of Writes and Split Completions 384
Initiator Must Generate Correct Data Phase Parity 384
Toggle Data and Parity When Target Inserts Wait State Pairs 384
In Reads，the Target Sources Data and Parity 384
First Data Phase Data and Parity Can Be Delayed 384
Subsequent Data Phase Data and Parity Never Delayed 384
General Discussion of Parity Checking 384
Checking Required in Address and Attribute Phases 384
Parity Checking Is Generally Required In Data Phases 385
Target Data Parity Checking During Write or Split Completion 385
Initiator Parity Checking During Reads 385
Parity Not Checked During First Data Phase Wait States 385
Parity Checked one Clock After Each Subsequent Data Phase 385
In Any Phase，Agent Driving AD Bus Supplies Parity 385
As in PCI，Even Parity Is Used 386
No Parity in Response Phase 386
Address Phase Parity 387
Address Phase Parity Checking Required 387
When DAC Is Used，Check Both Packets 387
On Error，SERR# Required 387
Error Detected Before Transaction Claimed 387
Error Detected After Transaction Claimed 388
Parity Error in Split Completion Address Phase 388
Attribute Phase Parity 389
Means Sequence ID and/or Byte Count Corrupted 389
When Attributes Corrupted and Split Response Issued 389
Data Phase Parity 389
Parity Always Covers Full Width of Data Bus 389
Parity Driven One Clock After Information Presented 390
Similar to PCI，But Different(due to registered bus) 390
Parity-Related Initiator Responsibilities 390
Initiator May Be a Requester，a Bridge，or a Completer 390
Requester′s Parity-Related Responsibilities During a Write 390
Drive Data，Byte Enables and the Parity 391
Requester Checks PERR 391
Requester Actions When PERR# Asserted by Target 391
Requester′s Actions on PERR# and Split Response 391
Bridge′s Parity-Related Responsibilities During a Write 391
Drive Data，Byte Enables and the Parity 392
Bridge Checks PERR 392
Bridge Actions When PERR# Asserted by Target 392
Initiator′s Parity-Related Responsibilities During a Split Completion 394
Who Initiates Split Completion Transactions? 394
Initiator Drives Data，Byte Enables and Parity 394
Initiator Does Not Monitor PERR 394
Initiator′s Parity-Related Responsibilities During a Read 395
General 395
First Read Data Item and Parity Can Be Delayed by Target 395
Subsequent Data Phases of a Read 395
Parity Checking by Initiator During a Read 395
Split Response Dummy Data Corrupted 396
Requester′s Parity-Related Responsibilities During a Read 396
Read Error and Data Parity Error Recovery Enable Bit=1 397
Read Error and Data Parity Error Recovery Enable Bit=0 397
Bridge′s Parity-Related Responsibilities During a Read 397
General 397
Bridge Handling of Parity Error in Immediate Read 397
Parity-Related Target Responsibilities 398
Target May Be a Completer，a Requester，or a Bridge 398
Target′s Responsibilities During a Write 398
When Completer Acts as the Target of the Write 399
When a Bridge Acts as the Target of the Write 399
Target′s Responsibilities During a Split Completion 399
When Requester Acts as the Target of a Split Completion 399
When a Bridge Acts as the Target of a Split Completion 400
Target′s Responsibilities During a Read 400
Data Parity Generation/Checking in PCI-X 401
General Format of Timing Diagram Descriptions 401
Parity in Memory Write Block(or Alias)or Split Completion 401
Parity in a Memory Write Transaction 406
Parity in Memory Read Block(or Alias)Transaction 406
Parity in Memory Read Dword or IO Read Transaction 411
Parity in Configuration Read 415
Parity in IO Write Transaction 416
Parity in Configuration Write 420
Part3：Device Configuration 425
Chapter 20：Configuration Transactions 425
Configuration Software Mechanism Same as PCI 425
Configuration Transactions Can Only Flow Downstream 426
Special Cycle Request Can Flow Upstream or Downstream 426
Type 0：Access Registers in Function on This Bus 426
Device Selection 426
Bus/Device Auto-Updated by Each Type 0 Config Write 430
General 430
Host/PCIX Bridge Device Number Assignment 430
Type 1：Access Registers in Function on Bus Farther out in Hierarchy 430
General Description 430
Some Example Scenarios 433
Example One 433
Example Two 433
Example Three 433
Example Four 434
Type 0 Configuration Transactions 435
Action Taken on Master Abort 435
IDSELs Routed Over Upper AD Lines 436
IDSEL Output Pins/Traces Allowed on Host/PCIX Bridge 437
Description 437
Type 0 Access by a Tool 438
Resistive Coupling Requires Address Pre-Drive 438
Bridge With IDSEL Pins Also Requires Address Pre-Drive 439
Type 0 Configuration Read 439
Type 0 Configuration Write 443
Type 1 Configuration Transactions 447
Although Not Necessary，Address Pre-Drive Used 447
Type 1 Configuration Access Address Format 447
Target Bus′s Source Bridge Converts Type 1 to Type 0 Access 448
Type 1 Transaction Examples 448
Arbiter′s Treatment of Configuration Access 449
Generation of Special Cycle Under Software Control 450
General 450
PCIX-to-PCIX Bridges Pass Special Cycle Request to Target Bus 451
Special Cycle by Device Other Than Host/PCIX Bridge 452
Chapter 21：Non-Bridge Configuration Registers 453
Detecting a PCI-X Capable Bridge 453
Detecting Presence of PCI-X Capable Bridge/Bus 453
Detecting Width of Bridge′s Interfaces 457
Detecting Frequency Support of Bridge′s Interfaces 458
Frequency Support of Bridge′s Primary Interface 458
Frequency Support on Bridge′s Secondary Interface 458
Checking Current Frequency/Protocol of Secondary Bus 459
Detecting Capabilities of Functions on Bus 459
Format of Non-Bridge Function′s PCI-X Capability Registers 459
Detecting PCI Functions 461
Detecting Width of PCI-X Functions 461
Detecting Frequency Support of PCI-X Functions 462
Most PCI Configuration Registers Remain Unchanged 462
Some PCI Config Registers Affected by Protocol Mode 463
Some Register Default Values Affected 463
Implementation of Some Registers Affected 463
Usage of Some Registers Affected 464
Base Address Registers(BARs) 464
Memory Base Address Registers 464
PCI and Memory BARs 464
Must Be Implemented as 64-bit Decoders 464
Definition of Prefetchable Memory 464
Minimum Memory Range 465
Diminishes Overall Number of BARs 465
IO Base Address Registers 467
Command Register Bits Affected by Protocol Mode 467
Introduction 467
Fast Back-to-Back Enable Bit 468
Stepping Control Bit 468
Memory Write and Invalidate Bit 468
Bus Master Bit 468
Status Register Bits Affected by Protocol Mode 469
Latency Timer Default Affected by Protocol Mode 469
Cache Line Size Configuration Register 470
Capabilities Pointer Register 470
Function′s PCI-X Capability Register Set 470
Register Set Format 470
Background on Load Tuning 471
Adjusting Requester′s Split Transaction Queue Size 471
Problem 471
Solution 471
Adjusting Requester′s Memory Read Transaction Size 471
Problem 471
Solution 472
PCI-X Command Register 472
Max Outstanding Split Transaction Field 473
Max Memory Read Byte Count Field 474
Enable Relaxed Ordering 474
RO Command Bit Enables/Disables Use of RO Attribute Bit 474
General Description 475
Usage in Memory Read 475
Usage in Memory Write 475
After Disconnect，Be Consistent 476
Restrictions on Use 476
For Additional Information 476
Data Parity Error Recover Enable 476
PCI-X Status Register 476
Bus，Device and Function Number Fields 477
Why They Are Necessary 477
Function Number Is Hardwired 477
Bus/Device Auto-Updated by Type 0 Configuration Write 477
Note for PCI-X Tool Designer 478
64-Bit Device Status Bit 479
133MHz-Capable Status Bit 479
Split Completion Discarded(this is not a good thing!) 480
Unexpected Split Completion(this is also not a good thing!) 480
Device Complexity 481
General 481
What Use Is This Bit? 481
Definition of a Simple Device 481
Rules Governing Behavior of a Simple Device 481
Designed Max Memory Read Byte Count 484
Designed Max Outstanding Split Transactions 484
Designed Max Cumulative Read Size 484
Chapter 22：Bridge Configuration Registers 487
Discovering a PCIX-to-PCIX Bridge 487
Many Bridge PCI Configuration Registers Unchanged 487
Some Bridge PCI Configuration Registers Affected by Mode 488
Command Register Affected 490
Status Register Affected 490
Capability Pointer Register Affected 490
Secondary Status Register Affected 490
General 490
Fast Back-to-Back Capable Bit Affected 491
Master Data Parity Error/Detected Parity Error Bits Affected 491
DEVSEL# Timing Bit Field 491
Cache Line Size Register Affected 491
Both Latency Timer Registers Affected 492
Base Address Registers(BARs)Affected 492
Secondary Bus Number Register Affected 493
In PCI Mode，No Change 493
In PCI-X Mode，Secondary Requesters May Have Pending Split Accesses 493
Hot-Plug Event May Cause Bus Renumbering 494
RST# Causes Functions to Forget Bus/Device Numbers 494
Prefetchable Base/Limit Registers Affected 495
Prefetchable Base/Limit Extension Registers Affected 495
Bridge Control Register Affected 496
Bridge′s PCI-X Capability Register Set 497
PCI-X Capability Register Set Format 497
PCI-X Secondary Status Register 498
64-Bit Device Status Bit 499
133MHz Capable Status Bit 499
Split Completion Discarded Bit 499
Unexpected Split Completion Bit 500
PCI Master Addresses PCI-X Target on Secondary 500
Problem：Bridge′s Requester ID，but Bad Tag 500
Error Handling 500
Bridge Secondary Interface Efficiency Status Bits 501
Background on Efficiency Status Bits 501
Split Completion Overrun Bit 501
Split Request Delayed Bit 502
Secondary Clock Frequency Bit Field 502
PCI-X Bridge(Primary Interface)Status Register 503
64-Bit Device Status Bit 503
133MHz Capable Status Bit 503
Split Completion Discarded Bit 504
Unexpected Split Completion Bit 504
PCI Master Addresses PCI-X Target on Primary 504
Problem：Bridge′s Requester ID，but Bad Tag 505
Error Handling 505
Bridge Primary Interface Efficiency Status Bits 505
Background on Efficiency Bits 505
Split Completion Overrun Bit 505
Split Request Delayed Bit 506
Bus/Device/Function Number Fields 506
Split Transaction Control Registers 506
Basic Register Format 506
The Bridge′s Split Completion Data Buffers 507
Controlling Bridge′s Use of Its Split Completion Data Buffers 507
Capacity Value Indicates Size of Respective Data Buffer 507
Limit Value Controls Bridge′s Perception of Buffer Size 508
Allocation Mode 508
Virtual Buffer Space Mode 509
Flood Mode 509
Regaining Control After Operating in Flood Mode 509
For More Information on Load Tuning 511
Optional Bridge Registers 511
Part4：Load Tuning 515
Chapter 23：Load Tuning Mechanisms 515
Introduction to Load Tuning 515
Non-Bridge Function Tuning 516
Information Fields 516
Adjustable Fields/Registers 516
Bridge Tuning 519
Adjusting Usage of Split Completion Buffers 519
Introduction 519
Interpreting the Efficiency Bits 519
Additional Spec Comments 519
Adjusting Bridge′s Timeslice Values 524
Bridge Is Surrogate for Initiators on Both Sides of Bridge 524
Software Must Assign Bridge Two Timeslices 525
Load-Tuning Software Can Choose Timeslice Other Than 64d 525
Part 5：PCI-X Bridges 532
Chapter 24：PCIX-to-PCIX Bridges 532
Performs Same Function as a PCI-to-PCI Bridge 532
Support for DAC Command 532
Downstream Movement of DAC optional for PCI Bridge 532
PCI-X Bridge Must Support Downstream DAC Movement 533
Bus Width 534
Memory Writes Crossing Bridge Are Always Posted 534
Other Transactions Crossing Bridge Are Always Split 534
How the Bridge Claims Split Completions 534
When Bridge Can Use Retry or Disconnect At Next ADB 535
When Bridge Can Issue a Retry 535
When Bridge Can Issue a Disconnect At Next ADB 536
Interfaces Can Be in Different Modes/Speeds 536
Translating PCI to PCI-X 536
General 536
Writes Are Posted 537
All Others Treated as PCI Delayed Transaction 537
On PCI Side，Bridge Follows PCI Ordering Rules 538
Translating the Transaction Type 538
Creating the Attributes 541
Attribute Bits 541
Requester ID 542
Transaction Tag 542
Byte Transfer Count 542
Target May Treat as Immediate or as Split Transaction 543
Translating PCI-X to PCI 543
General 543
Writes Are Posted 544
All Others Treated as Split Transactions 544
Translating the Transaction Type 544
PCI Target May Treat as Immediate or Delayed Transaction 547
Bridge Creation of a Split Completion 548
Effect of Relaxed Ordering Attribute Bit 549
Error Handling 549
Bridge Error Class SCMs 549
Error Handling Defined by Mode of Originating Interface 549
Error Handling When the Originating Bus Is in PCI-X Mode 550
Scenario 1：Data Parity Error in Immediate Read on Destination Bus 550
Scenario 2：Data Parity Error on Split Write 551
General 551
Scenario 2a 551
Scenario 2b 552
Scenario 2c 553
Scenario 2d 554
Scenario 3：Data Parity Error on Split Completion 555
Originating Bus Parity Error on Split Completion Read Data 555
Originating Bus Parity Error on SCM 555
Destination Bus Parity Error on Data or SCM 556
Bridge Forwards SCMs Quietly(without getting involved) 556
Scenario 4：Data Parity Error on Posted Memory Write 556
Scenario 5：Master Abort on Destination Bus 557
General 557
Master Abort When Forwarding Split Request 557
Master Abort on Posted Memory Write 558
Master Abort on Split Completion 559
Scenario 6：Target Abort on Destination Bus 559
Bridge Signals Target Abort 559
Bridge Receives A Target Abort on Split Transaction 559
Bridge Receives Target Abort on Memory Write 560
Bridge Receives Target Abort on Split Completion 560
Error Handling When Originating Bus in PCI Mode 560
Background 560
Bridge Handles Errors Same as PCI Bridge Unless SCM Error 561
SCM Error on a Split Read 561
SCM on Split Write 561
Corrupted Split Completion 562
Buffer Size 562
Posted Memory Write Buffer Size 562
Split Completion Data Buffer Size 563
Buffer Space for Memory Read Data 564
General 564
The Problem 564
When Limit=Capacity，This Is Not a Problem 564
When a Request Is Bigger Than the Bridge Buffer 564
Application Bridge 564
Bridge Acceptance Rules 567
Ordering and Passing Rules 568
Same Rules as PCI Except 569
Relaxed Ordering Effect on Transaction Ordering 569
Relaxed Ordering Effects on Memory Reads 569
Relaxed Ordering Effects on Memory Writes 570
Split Transaction Effects on Transaction Ordering 571
General 571
Same Sequence Split Completions Must Remain in Order 571
Relaxed Ordering Can Affect Split Completion Delivery 572
The Rules 572
Decomposing Split Transactions(sounds morbid!) 577
Chapter 25：Locked Transaction Series 579
Definition of Downstream and Upstream 579
Basics 580
Only Host/PCIX Bridge Originates Downstream Locked Series 581
PCI-X Bridges only Pass Locked Series Downstream 581
Only EISA Bridge Originates Upstream Locked Traffic 582
Application Bridge May or May Not Support Locking 582
EISA Bridge Supports LOCK# As Input，Not as Output 583
Non-Bridge Devices Ignore LOCK 583
Sequence of Events 583
Split Completion Error Message Terminates Lock 586
Upstream Bridge(Initiating Bridge)Rules 587
Downstream Bridge(Target Bridge)Rules 587
Arbitration 588
Starting Locked Transaction Series 588
Retry/Target Abort/Master Abort Cancels Lock 588
First Access of Series Receives Retry 588
General 588
If Target Is a PCI-to-PCI Bridge 588
If the Target Is a PCIX-to-PCIX Bridge 589
First Access of Series Receives Target or Master Abort 589
First Transaction Has Immediate Completion 589
First Transaction Receives Split Response 592
Continuing Locked Transaction Series 595
Attempted Access to Bridge by Device Other Than Owner 597
Last Transaction in Locked Series 598
If Last Transaction Receives Immediate Completion 598
If Last Transaction Receives Split Response 599
The Unlocking of a Target Bridge 599
When Bridge Starts Second Series Immediately After First 599
Retry Issued to Owner Not a Problem 599
DAC Command Lock Timing 600
In the First Access 600
In Subsequent Accesses of Locked Series 600
Part 6：Error Detection and Handling 603
Chapter 26：Error Detection and Handling 603
Handling of a Target Abort 603
Introduction 603
Requester Receives Target Abort 604
Completer Receives Target Abort on Split Completion 606
When Split Completion Target Abort Is Permissible 606
Discard of Completion for a Write or Non-Prefetchable Read 607
Discard of Completion for IO，Config，or Prefetchable Memory Read 608
Handling of a Master Abort 609
Introduction 609
Requester Handling of Master Abort 609
If Host/PCI Bridge Type 0 Config Transaction 609
If Not Host/PCIX Bridge Type 0 Config Transaction 610
If Requester Capable of Generating Interrupts 610
If Requester Can′t Generate Interrupts 610
Completer Handling of Master Abort on a Split Completion 610
General 610
Discard of Completion for a Write or Non-Prefetchable Read 611
Discard of Completion of IO，Config，or Prefetchable Memory Read 611
Target Handling of Address or Attribute Phase Parity Error 612
Data Phase Parity 612
PCI Chipset Typically Murders System on Data Parity Error 612
In PCI-X，Chipset Doesn′t Monitor PERR 613
General 613
Requester′s Data Parity Error Recovery Enable Bit 613
Basic Description 613
Set by OS or Driver′s Initialization Code at Startup Time 614
If Not Set，PCI Compatibility Dictates Murdering System 614
Recovery Must Be Performed Under Software Guidance 615
Driver Reports Error to OS 615
OS Instructs Driver on Action(s)to Take 615
OS Vendor Specifies Who Clears Parity Error Status Bits 615
Requester Handling of PERR# With Split Response 616
Data Error Received With Split Response on Read 616
Data Error Received With Split Response on Dword Write 618
Requester/Completer Handling of Data Error During SPlit Completion 620
Split Read Errors 620
General 620
Read Data Returned in One or More Split Completions 621
Internal Completer Error Can Occur at Any Time 622
Receipt of SCM Terminates Read Sequence 622
Effect of SCM on a Bridge 623
Effect of SCM on the Requester 623
Requester Handling of Unexpected Split Completion 626
Requester Handling of Split Completion Error Messages 627
Handling Completer Byte Count Out-of-Range Error 627
Memory Writes Never Split，so SCMs Don′t Apply 627
Read Immediate Treated Same as Write at Device Boundary 627
Split Read Straddling Device Boundary Causes SCM 627
Read Burst Straddling Device Boundary Considered Rare Case 628
Handling Completer Split Write Data Parity Error 629
Handling Completer Device-Specific Error 629
Handling Master Abort on Other Side of a Bridge 629
Error Description 629
Effect of Master Abort on Bridge 630
Bridge Interrupt Acknowledge Master Aborts 630
Bridge Special Cycle Master Aborts 630
Bridge IO Read Master Aborts 630
Bridge IO Write Master Aborts 632
Bridge Configuration Read Master Aborts 632
Bridge Configuration Write Master Aborts 632
Bridge Memory Read Dword Master Aborts 632
Bridge Memory Write Master Aborts 632
Bridge Memory Read Ends in Master Abort 633
Bridge Split Completion Ends in Master Abort 633
Effect of Master Abort SCM on Requester 635
Requester Interrupt Acknowledge Gets Master Abort SCM 635
Requester Special Cycle Gets Master Abort SCM 635
Requester IO Read or IO Write Gets Master Abort SCM 636
Requester Configuration Read Gets Master Abort SCM 636
Requester Configuration Write Gets Master Abort SCM 636
Requester Memory Read Dword Gets Master Abort SCM 637
Requester Memory Write Gets Master Abort SCM 637
Requester Memory Read Gets Master Abort SCM 637
Requester Split Completion Gets Master Abort SCM 638
Handling Target Abort on Other Side of Bridge 638
Error Description 638
Effect of Target Abort on Bridge 638
Bridge Interrupt Acknowledge Target Aborts 639
Bridge Special Cycle Target Aborts 639
Bridge IO Read Target Aborts 639
Bridge IO Write Target Aborts 640
Bridge Configuration Read Target Aborts 640
Bridge Configuration Write Target Aborts 640
Bridge Memory Read Dword Target Aborts 640
Bridge Memory Write Target Aborts 640
Bridge Memory Read Block Ends in Target Abort 641
Bridge Split Completion Ends in Target Abort 641
Effect of Target Abort SCM on Requester 642
Requester Interrupt Acknowledge Gets Target Abort SCM 642
Requester Special Cycle Gets Target Abort SCM 642
Requester IO Read or IO Write Gets Target Abort SCM 642
Requester Configuration Read Gets Target Abort SCM 643
Requester Configuration Write Gets Target Abort SCM 643
Requester Memory Read Dword Gets Target Abort SCM 643
Requester Memory Write Gets Target Abort SCM 644
Requester Memory Read Gets Target Abort SCM 644
Requester Split Completion Gets Target Abort SCM 644
Handling Split Write Data Parity Error on Other Side of Bridge 644
Error Description 644
Bridge May Deliberately Forward a Bad Split Write 644
…Or Split Write May Become Corrupted When Bridge Forwards It 645
Effect of Split Write Data Parity Error on Bridge 645
Effect of Split Write Data Parity Error on Requester 645
Usage of SERR 646
Part 7： Electrical Issues 651
Chapter 27：Electrical Issues 651
Introduction 651
LVS Bus 651
Attention to Detail 651
Most Parameters Tighter Than PCI 652
Maximum Number of Loads and Connectors on Bus 652
Cards Keyed as 3.3 Volt or Universal Cards 652
133MHz PCI-X Device Must Support 66MHz PCI 652
Add-In Card Trace Lengths 652
Initialization Pattern Setup/Hold Time 653
General 653
Trhff Must Be Taken Into Account 653
IDSEL Routing 654
General 654
IDSEL Series Resistor Value 655
Appendix A：Protocol Rules 659
Introduction 659
General Bus Rules 659
Initiator Rules 661
Target Rules 662
Bus Arbitration Rules 663
Configuration Transaction Rules 664
Parity Error Rules 665
Bus Width Rules 665
Split Transaction Rules 666
Appendix B：Glossary 671
Index 683