Difference between revisions of "APF6 SP The full howto"
(→Adding the DDR3) |
(→Generate bitstream with Quartus) |
||
(120 intermediate revisions by 2 users not shown) | |||
Line 1: | Line 1: | ||
[[Category: APF6]] | [[Category: APF6]] | ||
+ | [[Category: APF6_SP]] | ||
[[Category: FPGA]] | [[Category: FPGA]] | ||
[[Category: CycloneV]] | [[Category: CycloneV]] | ||
+ | [[Category: Quartus]] | ||
+ | [[Category: Qsys]] | ||
− | = Introduction = | + | == Introduction == |
− | This is the full howto | + | This is the full howto make an APF6_SP CycloneV example design which uses : |
* PCIe | * PCIe | ||
* DDR3 | * DDR3 | ||
− | * LED | + | * General Purpose Input/Outputs |
− | * Button | + | ** LED |
+ | ** Button | ||
This howto is designed for : | This howto is designed for : | ||
− | * APF6D_SP-M1GE4G-4m-BW-C4M384: APF6_SP Dual with CycloneV C4 (speed C8) and two DDR3 chips | + | * [[APF6_SP|APF6D_SP-M1GE4G-4m-BW-C4M384]]: APF6_SP Dual with CycloneV C4 (speed C8) and two DDR3 chips |
− | * Quartus | + | * Quartus with following versions: |
+ | ** '''Quartus 17.0.0''' : Quartus 17 without update | ||
+ | ** '''Quartus 19.1''' : Quartus Intel version | ||
+ | ** To install Quartus see [[Quartus_installation_on_Linux | this page]]. | ||
[[File:APF6_SP_full_howto.png|frame|center|The full design for this howto]] | [[File:APF6_SP_full_howto.png|frame|center|The full design for this howto]] | ||
− | = | + | = Make the quartus project = |
− | + | * Open the "New Project Wizard" in Quartus | |
+ | * Select a directory for the design, for example ''/home/your_name/workspace'' and a project name ''apf6_the_full_howto'': | ||
+ | * Then Choose ''Empty Project'' and don't add any file yet -> press "Next" | ||
+ | * Select the part number : '''5CGXFC4C7U19C8''' -> press "Next" -> "Next" -> "Finish" | ||
+ | * Right click on "Cyclone V -> Devices..." in project navigator and click on '''Device and Pin Options''' | ||
+ | * in '''CvP Settings''' set this options: | ||
+ | ** Configuration via Protocol: '''Core initialization and update''' | ||
+ | * then select "OK" two times | ||
− | + | = Make the Qsys project = | |
− | * Open | + | * Open Qsys/Platform designer: |
− | + | ** With Quartus 17: '''Tools'''->'''Qsys''' | |
− | + | ** With Quartus 19.1: '''Tools'''->'''Platform designer''' | |
− | + | * Delete the component '''clk_0''' then save the Qsys project as : '''qsys_tfht.qsys''' | |
− | + | * Click on '''Finish''' and generate the component with all options left by default | |
− | * | + | * Under Quartus project add the Qsys project (don't forget to click on '''add''' after selected here). |
− | * | + | |
− | + | ||
− | * | + | |
− | * Delete the component '''clk_0''' then save the Qsys project | + | |
− | * | + | |
− | * Under Quartus project add the Qsys project (don't forget to click on '''add''' after selected here. | + | |
[[File:tfht_add_qsys_project.png|frame|center|How to add a file in project]] | [[File:tfht_add_qsys_project.png|frame|center|How to add a file in project]] | ||
− | * Relaunch Qsys by double- | + | * Right click on ''qsys_tfht.qsys'' and '''set as Top-Level Entity''' |
+ | * Relaunch Qsys by double-clicking on the file in "Project Navigator" | ||
== Adding the PCIe and CvP == | == Adding the PCIe and CvP == | ||
Line 45: | Line 54: | ||
Interface Protocols | Interface Protocols | ||
PCI Express | PCI Express | ||
− | Avalon-MM Cyclone V Hard IP for PCI Express | + | Avalon-MM Cyclone V Hard IP for PCI Express |
+ | Avalon-MM Cyclone V Hard IP for PCI Express Intel FPGA IP <- 19.1 | ||
</pre> | </pre> | ||
+ | * "Add" it to design. | ||
* PCIe configuration: | * PCIe configuration: | ||
** Number of Lanes: '''x1''' | ** Number of Lanes: '''x1''' | ||
** Reference clock frequency: '''125Mhz''' | ** Reference clock frequency: '''125Mhz''' | ||
− | ** check | + | ** check '''Use 62.5 MHz application clock''' |
+ | ** check ''' Enable configuration via the PCIe link''' | ||
** BAR0: 32-bit non-prefetchable memory | ** BAR0: 32-bit non-prefetchable memory | ||
** BAR1: 32-bit non-prefetchable memory | ** BAR1: 32-bit non-prefetchable memory | ||
Line 58: | Line 70: | ||
** Class Code: '''0x00001300''': there is a bug in quartus, the actual class code is here '''13''' | ** Class Code: '''0x00001300''': there is a bug in quartus, the actual class code is here '''13''' | ||
− | * click on | + | * click on "Finish" |
− | * Export these | + | * Export these signals with name in '''bold''' (right click in "Hierarchy", then choose "Connections" and "Export as:..."). Exported signals will be accessible on top of module: |
** refclk : '''refclk''' | ** refclk : '''refclk''' | ||
** npor: '''npor''' | ** npor: '''npor''' | ||
** hip_serial: '''hip_serial''' | ** hip_serial: '''hip_serial''' | ||
− | * connect '''Rxm_BAR0''' on '''Cra''' | + | * connect '''Rxm_BAR0''' on '''Cra''' (always right click in "Hierarchy", then "Connections"...) |
− | * | + | * Add CvP component reconfig driver, with all default parameters: |
<pre class="config"> | <pre class="config"> | ||
Library | Library | ||
Line 71: | Line 83: | ||
PCI Express | PCI Express | ||
Altera PCIe Reconfig Driver | Altera PCIe Reconfig Driver | ||
+ | Altera PCIe Reconfig Driver Intel FPGA IP <- 19.1 | ||
</pre> | </pre> | ||
− | * Connect | + | * Connect previous IP signals with: |
** '''coreclkout''' -> '''reconfig_xcvr_clk''' | ** '''coreclkout''' -> '''reconfig_xcvr_clk''' | ||
** '''coreclkout''' -> '''pld_clk''' | ** '''coreclkout''' -> '''pld_clk''' | ||
** '''nreset_status''' -> '''reconfig_xcvr_rst''' | ** '''nreset_status''' -> '''reconfig_xcvr_rst''' | ||
− | * | + | * Add CvP component transceiver, with all default parameters: |
<pre class="config"> | <pre class="config"> | ||
Library | Library | ||
Line 83: | Line 96: | ||
Transceiver PHY | Transceiver PHY | ||
Transceiver Reconfiguration Controller | Transceiver Reconfiguration Controller | ||
+ | Transceiver Reconfiguration Controller Intel FPGA IP <- 19.1 | ||
</pre> | </pre> | ||
− | * Connect | + | * Connect previous PCIe IP signals with: |
** '''coreclkout''' -> '''mgmt_clk_clk''' | ** '''coreclkout''' -> '''mgmt_clk_clk''' | ||
** '''nreset_status''' -> '''mgmt_rst_reset''' | ** '''nreset_status''' -> '''mgmt_rst_reset''' | ||
− | ** '''reconfig_mgmt''' -> '''reconfig_mgmt''' | + | ** reconfig_driver.'''reconfig_mgmt''' -> '''reconfig_mgmt''' |
** reconfig_driver.'''reconfig_busy''' -> '''reconfig_busy''' | ** reconfig_driver.'''reconfig_busy''' -> '''reconfig_busy''' | ||
** pci.'''reconfig_to_xcvr''' -> '''reconfig_to_xcvr''' | ** pci.'''reconfig_to_xcvr''' -> '''reconfig_to_xcvr''' | ||
** pci.'''reconfig_from_xcvr''' -> '''reconfig_from_xcvr''' | ** pci.'''reconfig_from_xcvr''' -> '''reconfig_from_xcvr''' | ||
+ | * you should end up with the following wiring: | ||
[[File:APF6_tfht_pcie_wiring.png|frame|center|Wiring result after PCIe configured]] | [[File:APF6_tfht_pcie_wiring.png|frame|center|Wiring result after PCIe configured]] | ||
− | |||
== Adding the DDR3 == | == Adding the DDR3 == | ||
− | {{Note| | + | {{Note| If not indicated, parameters should be left to default one !}} |
+ | |||
+ | === UniPHY controller === | ||
* In the IP catalog, select: | * In the IP catalog, select: | ||
Line 105: | Line 121: | ||
Memory Interfaces with UniPHY | Memory Interfaces with UniPHY | ||
DDR3 SDRAM Controller with UniPHY | DDR3 SDRAM Controller with UniPHY | ||
+ | DDR3 SDRAM Controller with UniPHY Intel FPGA IP <- 19.1 | ||
</pre> | </pre> | ||
Line 116: | Line 133: | ||
** supply Voltage: '''1.35V DDR3L''' | ** supply Voltage: '''1.35V DDR3L''' | ||
* In tab '''Memory Parameters''' | * In tab '''Memory Parameters''' | ||
− | ** In | + | ** In "Presets" (right of the window) select: '''JEDEC DDR3-1G6 2GB X8''' and apply |
** Memory device speed grade: '''400Mhz''' | ** Memory device speed grade: '''400Mhz''' | ||
** Total interface width: '''24''' | ** Total interface width: '''24''' | ||
Line 123: | Line 140: | ||
* In tab '''Controller Settings''' | * In tab '''Controller Settings''' | ||
** checkbox: '''Generate power-of-2 data bus widths for Qsys or SOPC Builder''' | ** checkbox: '''Generate power-of-2 data bus widths for Qsys or SOPC Builder''' | ||
+ | ** checkbox: '''Enable Configuration and Status Register Interface''' (Internal JTAG) | ||
** checkbox: '''Enable Error Detection and Correction Logic''' | ** checkbox: '''Enable Error Detection and Correction Logic''' | ||
− | * Click on ''finish'' | + | * Click on ''Finish'' |
+ | |||
+ | * Connect following signals: | ||
+ | ** '''coreclkout''' -> '''pll_ref_clk''' | ||
+ | ** '''coreclkout''' -> '''mp_cmd_clk_0''' | ||
+ | ** '''coreclkout''' -> '''mp_rfifo_clk_0''' | ||
+ | ** '''coreclkout''' -> '''mp_wfifo_clk_0''' | ||
+ | ** '''coreclkout''' -> '''csr_clk''' | ||
+ | ** '''nreset_status''' -> '''global_reset''' | ||
+ | ** '''nreset_status''' -> '''soft_reset''' | ||
+ | ** '''nreset_status''' -> '''mp_cmd_reset_n_0''' | ||
+ | ** '''nreset_status''' -> '''mp_rfifo_reset_n_0''' | ||
+ | ** '''nreset_status''' -> '''mp_wfifo_reset_n_0''' | ||
+ | ** '''nreset_status''' -> '''csr_reset_n''' | ||
+ | |||
+ | === Span extender === | ||
+ | |||
+ | DDR3 memory is to large for PCIe BAR domain. Then we have to use a [https://documentation.altera.com/#/00030683-AA$NT00064780 span extender] to adapt PCIe BAR domain to DDR3 domain. | ||
+ | |||
+ | * Select span expander in IP Library: | ||
+ | <pre class="config"> | ||
+ | Library | ||
+ | Basic Functions | ||
+ | Bridges and Adaptors | ||
+ | Memory Mapped | ||
+ | Address Span Extender | ||
+ | </pre> | ||
+ | * Configure it with these options: | ||
+ | ** Datapath Width: '''64bits''' | ||
+ | ** Slave Word Address Width: '''20bits''' | ||
+ | ** press "Finish" | ||
+ | |||
+ | * Connect following PCIe IP signals to extender one: | ||
+ | ** '''coreclckout''' -> '''clock''' | ||
+ | ** '''nreset_status''' -> '''reset''' | ||
+ | ** '''Rxm_BAR0''' -> '''cntl''' | ||
+ | *** Change base address of cntl to '''0x0000_4000''' | ||
+ | ** '''Rxm_BAR1''' -> '''windowed_slave''' | ||
+ | * And following extender signals to DDR3 one: | ||
+ | ** '''expanded_master''' -> mem_if_ddr3_emif_0.'''avl_0''' | ||
+ | |||
+ | == Adding the button and LED (PIO) == | ||
+ | |||
+ | Documentation of the PIO component can be found [https://www.altera.co.jp/content/dam/altera-www/global/ja_JP/pdfs/literature/hb/nios2/n2cpu_nii51007.pdf here]. | ||
+ | |||
+ | * In IP Library select: | ||
+ | <pre class="config"> | ||
+ | Library | ||
+ | Processors and Peripherals | ||
+ | Peripherals | ||
+ | PIO (Parallel I/O) | ||
+ | PIO (Parallel I/O) Intel FPGA IP <- 19.1 | ||
+ | </pre> | ||
+ | * Configure it with these options: | ||
+ | ** Width: '''2''' | ||
+ | ** Direction: '''Bidir''' | ||
+ | ** checkbox '''Synchronously capture''' | ||
+ | ** Edge Type: '''RISING''' | ||
+ | ** checkbox '''Generate IRQ''' | ||
+ | ** IRQ Type: '''EDGE''' | ||
+ | * click on "Finish" | ||
+ | |||
+ | * export ''pio.external_connection'' to '''pio''' | ||
+ | |||
+ | * Connect following signals: | ||
+ | ** '''coreclkout''' -> '''pio_0.clk''' | ||
+ | ** '''nreset_status''' -> '''pio_0.reset''' | ||
+ | ** '''Rxm_BAR2''' -> '''pio_0.s1''' | ||
+ | ** '''pio_0.irq''' -> '''Rxmirq''' | ||
+ | |||
+ | == Qsys component schematic == | ||
+ | |||
+ | The resulting schematics should give the figure below. | ||
+ | |||
+ | [[File:apf6_tfht_pcieddr_sch.png|frame|center| The full connections in qsys component]] | ||
+ | |||
+ | Now click on '''finish''' and generate the component with all default parameters kept. | ||
+ | |||
+ | = Generate bitstream with Quartus = | ||
+ | |||
+ | * Make a first synthesis, in ''Task'' window, right click on ''Analysis & Synthesis'' then choose ''Start'' | ||
+ | <pre class="config"> | ||
+ | Compile Design | ||
+ | + Analysis & Synthesis | ||
+ | </pre> | ||
+ | {{Note| If you have a synthesis error here with 19.1 be sure that you are using your [[Quartus_installation_on_Linux#Perl_Getopt::Long | perl distribution]].}} | ||
+ | == Pinout == | ||
+ | |||
+ | * PCIe | ||
+ | ** Outside Quartus, create a file named '''pcie_pinout.tcl'' and edit it with your favorite text editor. | ||
+ | ** Copy all lines from [[APF6_SP_DDR3_PINOUT#PCIe| here]]. | ||
+ | ** Save your file, then add it to the project inside Quartus | ||
+ | ** Run the script with following menu, select it then '''run''': | ||
+ | <pre class="config"> | ||
+ | Tools | ||
+ | TCL Scripts ... | ||
+ | </pre> | ||
+ | * DDR3 | ||
+ | ** Create a file named '''ddr3_pinout.tcl''' and edit it. | ||
+ | ** Copy all lines from [[APF6_SP_DDR3_PINOUT#DDR3| here]] ('''Pinout placement''' and '''Technology pinout'''). | ||
+ | ** Save then add the file to project | ||
+ | ** Run the script with following menu, select it then '''run''': | ||
+ | <pre class="config"> | ||
+ | Tools | ||
+ | TCL Scripts ... | ||
+ | </pre> | ||
+ | * Button & Led | ||
+ | ** Open the pinplanner by double-click in '''tasks''': | ||
+ | <pre class="config"> | ||
+ | Compile Design | ||
+ | Analysis & Synthesis | ||
+ | I/O Assignment Analysis | ||
+ | Pin Planner | ||
+ | </pre> | ||
+ | ** For button '''pio_export[0]''' select Location column and set it to : '''PIN_L9''' | ||
+ | ** For led '''pio_export[1]''' select Location column and set it to : '''PIN_M10''' | ||
+ | ** Close the Pin Planner | ||
+ | |||
+ | == Synthesis, Place & route == | ||
+ | |||
+ | * Right click on ''Analysis and Synthesis'' then '''Start''' | ||
+ | * Open a Linux console terminal then go to your project directory and type this: | ||
+ | <pre class="host"> | ||
+ | $ cd ~/tmp/apf6_the_full_howto | ||
+ | $ find . -name "*_p0_pin_map.tcl" | xargs sed -iolt "/_mem_if_ddr3_emif_0_p0_get_input_clk_id {/a return \$pll_output_node_id" | ||
+ | </pre> | ||
+ | {{Note| This is the [[DDR3-CycloneV_interface_description#The_DDR3_clock_hack | DDR3 clock hack]], each time synthesis is relaunched, this shell line must be relaunch}} | ||
+ | * Right click on ''Assembler (Generate programming files)'' then '''Start''' | ||
+ | * Convert bitstream to rbf with menu: | ||
+ | <pre class="config"> | ||
+ | File | ||
+ | Convert Programming Files... | ||
+ | </pre> | ||
+ | * set these options: | ||
+ | ** Programming file type: '''Raw Binary File (.rbf)''' | ||
+ | ** Change the name to : '''output_files/tfht.rbf''' | ||
+ | ** Select '''SOF Data''' in '''Input files to convert''' then click on '''Add File...''' | ||
+ | ** add the file '''apf6_the_full_howto.sof''' in directory ''output_files'' | ||
+ | ** checkbox '''Create CvP files (Generate tfht.periph.rbf and tfht.core.rbf)''' | ||
+ | ** click on '''Generate''' | ||
+ | * These files should be present on directory '''output_files''': | ||
+ | <pre class="host"> | ||
+ | ls -l tmp/apf6_the_full_howto/output_files/*.rbf | ||
+ | tmp/apf6_the_full_howto/output_files/tfht.core.rbf | ||
+ | tmp/apf6_the_full_howto/output_files/tfht.periph.rbf | ||
+ | </pre> | ||
+ | |||
+ | {{Note| These bitstreams can be found under armadeus distribution in directory `firmware/tfht/bitstreams` }} | ||
+ | |||
+ | = Linux BSP configuration = | ||
+ | |||
+ | {{Note| This is not an Armadeus BSP tutorial, the BSP is supposed to be installed and compiled according to this [[Toolchain| explains]]}} | ||
+ | |||
+ | * Get a compiled BSP with following configuration : '''apf6_defconfig''' | ||
+ | * Add [[Pci_debug| PCI debug]] package with '''make menuconfig''' | ||
+ | <pre class="config"> | ||
+ | Target packages ---> | ||
+ | Hardware handling ---> | ||
+ | [*] pcidebug | ||
+ | </pre> | ||
+ | {{Note | On current Armadeus BSP, pcidebug is already installed by default}} | ||
+ | * Then make the binaries and [[Target_Software_Installation |flash it]] on the apf6_sp. | ||
+ | * Adding the rbf file to your [[Communicate_with_your_board_from_a_Linux_Host_(Basics)#TFTP_server | tftpboot]] directory. | ||
+ | <pre class="host"> | ||
+ | $ cp -v output_files/tfht.*.rbf /tftpboot/ | ||
+ | ‘output_files/tfht.core.rbf’ -> ‘/tftpboot/tfht.core.rbf’ | ||
+ | ‘output_files/tfht.periph.rbf’ -> ‘/tftpboot/tfht.periph.rbf’ | ||
+ | </pre> | ||
+ | |||
+ | {{Note| Refer to [[PCIe FPGA loading]] to understand FPGA configuration with PCIe}} | ||
+ | * In U-Boot download the periph.rbf and configure FPGA (don't forget to configure your tftp server ip): | ||
+ | <pre class="apf"> | ||
+ | BIOS> tftpboot ${loadaddr} 192.168.0.214:tfht.periph.rbf | ||
+ | BIOS> fpga load 0 ${loadaddr} ${filesize} | ||
+ | </pre> | ||
+ | * Boot Linux | ||
+ | <pre class="apf"> | ||
+ | BIOS> boot | ||
+ | ... | ||
+ | apf6 login: root | ||
+ | </pre> | ||
+ | |||
+ | * The three BARx should be seen with '''lspci''' command : | ||
+ | <pre class="apf"> | ||
+ | # lspci -v -s 01:00.0 | ||
+ | 01:00.0 Unclassified device [0013]: Altera Corporation Device e001 (rev 01) | ||
+ | Flags: fast devsel, IRQ 341 | ||
+ | Memory at 01800000 (32-bit, non-prefetchable) [disabled] [size=32K] | ||
+ | Memory at 01000000 (32-bit, non-prefetchable) [disabled] [size=8M] | ||
+ | Memory at 01808000 (32-bit, non-prefetchable) [disabled] [size=32] | ||
+ | Capabilities: [50] MSI: Enable- Count=1/4 Maskable- 64bit+ | ||
+ | Capabilities: [78] Power Management version 3 | ||
+ | Capabilities: [80] Express Endpoint, MSI 00 | ||
+ | Capabilities: [100] Virtual Channel | ||
+ | Capabilities: [200] Vendor Specific Information: ID=1172 Rev=0 Len=044 <?> | ||
+ | </pre> | ||
+ | * Configure network: | ||
+ | <pre class="apf"> | ||
+ | $ udhcpc | ||
+ | </pre> | ||
+ | * Download core: | ||
+ | <pre class="apf"> | ||
+ | $ tftp -g -r tfht.core.rbf 192.168.0.214 | ||
+ | </pre> | ||
+ | * Load the core bitstream: | ||
+ | <pre class="apf"> | ||
+ | $ load_fpga tfht.core.rbf | ||
+ | Altera CvP 0000:01:00.0: enabling device (0140 -> 0142) | ||
+ | Altera CvP 0000:01:00.0: Found and enabled PCI device with VID 0x1172, DID 0xE001 | ||
+ | successfully init Altera CVP with major 248 | ||
+ | Altera CvP 0000:01:00.0: Now starting CvP... | ||
+ | Altera CvP 0000:01:00.0: CvP successful, application layer now ready | ||
+ | 8212+1 records in | ||
+ | 8212+1 records out | ||
+ | </pre> | ||
+ | |||
+ | == User access to FPGA == | ||
+ | |||
+ | === DDR3 === | ||
+ | |||
+ | * DDR3 is accessible via the BAR1, connect to BAR1: | ||
+ | <pre class="apf"> | ||
+ | $ pci_debug -s 01:00.0 -b1 | ||
+ | </pre> | ||
+ | |||
+ | * Read first values of DDR3: | ||
+ | <pre class="apf"> | ||
+ | PCI> d32 0 20 | ||
+ | |||
+ | 00000000: AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA | ||
+ | 00000010: 55555555 55555555 55555555 55555555 | ||
+ | </pre> | ||
+ | |||
+ | * Write values, then read back: | ||
+ | <pre class="apf"> | ||
+ | PCI> c32 0 CAFEDECA | ||
+ | PCI> c32 8 CACABEBE | ||
+ | PCI> d32 0 20 | ||
+ | |||
+ | 00000000: CAFEDECA AAAAAAAA CACABEBE AAAAAAAA | ||
+ | 00000010: 55555555 55555555 55555555 55555555 | ||
+ | </pre> | ||
+ | |||
+ | * Exit pci_debug | ||
+ | <pre class="apf"> | ||
+ | PCI> quit | ||
+ | </pre> | ||
+ | |||
+ | Only first 8MB (0x800000) are accessible for the moment, to access next 8MB, the span extender must be configured on BAR0. | ||
+ | |||
+ | * Access BAR0 and configure [https://documentation.altera.com/#/00030683-AA$NT00064780 span extender]: | ||
+ | <pre class="apf"> | ||
+ | $ pci_debug -s 01:00.0 -b0 | ||
+ | </pre> | ||
+ | |||
+ | * Read the span extender windows value at address 0x4000: | ||
+ | <pre class="apf"> | ||
+ | PCI> d32 4000 1 | ||
+ | |||
+ | 00004000: 00000000 | ||
+ | </pre> | ||
+ | |||
+ | * Write the new required value : | ||
+ | <pre class="apf"> | ||
+ | PCI> c32 4000 800000 | ||
+ | PCI> d32 4000 1 | ||
+ | |||
+ | 00004000: 00800000 | ||
+ | </pre> | ||
+ | |||
+ | * Quit: | ||
+ | <pre class="apf"> | ||
+ | PCI> quit | ||
+ | </pre> | ||
+ | * then reconnect pci_debug to change for bar1: | ||
+ | <pre class="apf"> | ||
+ | $ pci_debug -s 01:00.0 -b1 | ||
+ | </pre> | ||
+ | |||
+ | * Read value : | ||
+ | <pre class="apf"> | ||
+ | PCI> d32 0 20 | ||
+ | |||
+ | 00000000: 6B2171EA DFA5DA76 6CFB3F47 7AF9F526 | ||
+ | 00000010: DBD85BAB 7F56CF7A 78FAD73B ECF997FE | ||
+ | </pre> | ||
+ | |||
+ | * Write some values: | ||
+ | <pre class="apf"> | ||
+ | PCI> c32 0 12345678 | ||
+ | PCI> c32 4 9ABCDEF0 | ||
+ | PCI> c32 10 12345678 | ||
+ | PCI> c32 14 9ABCDEF0 | ||
+ | PCI> d32 0 20 | ||
+ | |||
+ | 00000000: 12345678 9ABCDEF0 6CFB3F47 7AF9F526 | ||
+ | 00000010: 12345678 9ABCDEF0 78FAD73B ECF997FE | ||
+ | </pre> | ||
+ | |||
+ | * Etc... | ||
+ | |||
+ | === PIO (Button and led) === | ||
+ | |||
+ | As seen on the first schematic of this tutorial, the [https://documentation.altera.com/#/00081203-NT$NT00080322 PIO] is plugged on the BAR2 of the PCIe. | ||
+ | |||
+ | * Then first, connect to BAR2 with pci_debug: | ||
+ | <pre class="apf"> | ||
+ | $ pci_debug -s 01:00.0 -b2 | ||
+ | </pre> | ||
+ | |||
+ | '''Button''' | ||
+ | * Read all PIO registers: | ||
+ | <pre class="apf"> | ||
+ | PCI> d32 0 8 | ||
+ | |||
+ | 00000000: 00000001 00000000 | ||
+ | </pre> | ||
+ | |||
+ | * Leave button pressed and read again: | ||
+ | <pre class="apf"> | ||
+ | PCI> d32 0 8 | ||
+ | |||
+ | 00000000: 00000000 00000000 | ||
+ | </pre> | ||
+ | |||
+ | '''led''' | ||
+ | * set PIO[1] (led) as output: | ||
+ | <pre class="apf"> | ||
+ | PCI> c32 4 2 | ||
+ | </pre> | ||
+ | |||
+ | * Turn on led | ||
+ | <pre class="apf"> | ||
+ | PCI> c32 0 2 | ||
+ | </pre> | ||
+ | |||
+ | * Turn off led: | ||
+ | <pre class="apf"> | ||
+ | PCI> c32 0 0 | ||
+ | </pre> | ||
+ | |||
+ | = Conclusion = | ||
+ | |||
+ | This tutorial should give you the basics to use the CycloneV soldered on the APF6_SP module. Now you should know: | ||
+ | |||
+ | * PCIe: | ||
+ | ** How to instanciate the CycloneV hard IP | ||
+ | ** How to read/write data with it from Linux with simple access | ||
+ | |||
+ | * DDR3: | ||
+ | ** How to instanciate and configure the UniPHY hard IP | ||
+ | ** How to connect it to PCIe | ||
+ | ** How to read/write simple data | ||
+ | |||
+ | * Simple I/O: | ||
+ | ** How to instanciate a simple port for I/O under Qsys | ||
+ | ** How to drive it from Linux | ||
+ | |||
+ | Some subjects '''are not covered by''' this tutorial: | ||
+ | |||
+ | * Managing PCIe interrupts | ||
+ | * Write a Linux driver | ||
+ | * Increase data bandwidth with PCIe-DMA (up to 140MB/s) | ||
+ | ** From the FPGA to the i.MX6 | ||
+ | ** From the i.MX6 to the FPGA | ||
+ | * Create a custom Qsys component | ||
+ | * Using Serializer/Deserializer | ||
+ | * And more ... | ||
+ | |||
+ | |||
+ | If you are interested in a specific subject, please contact [http://www.opossom.com/english/contact.html Armadeus System] they should do somethings for you. | ||
+ | |||
+ | = Links & Resources= | ||
+ | |||
+ | * [http://www.armadeus.com/_downloads/apf6_SP/hardware/apf6_the_full_howto.zip The full archive of quartus 15.1.1 project](zip) |
Latest revision as of 14:56, 12 November 2019
Contents
Introduction
This is the full howto make an APF6_SP CycloneV example design which uses :
- PCIe
- DDR3
- General Purpose Input/Outputs
- LED
- Button
This howto is designed for :
- APF6D_SP-M1GE4G-4m-BW-C4M384: APF6_SP Dual with CycloneV C4 (speed C8) and two DDR3 chips
- Quartus with following versions:
- Quartus 17.0.0 : Quartus 17 without update
- Quartus 19.1 : Quartus Intel version
- To install Quartus see this page.
Make the quartus project
- Open the "New Project Wizard" in Quartus
- Select a directory for the design, for example /home/your_name/workspace and a project name apf6_the_full_howto:
- Then Choose Empty Project and don't add any file yet -> press "Next"
- Select the part number : 5CGXFC4C7U19C8 -> press "Next" -> "Next" -> "Finish"
- Right click on "Cyclone V -> Devices..." in project navigator and click on Device and Pin Options
- in CvP Settings set this options:
- Configuration via Protocol: Core initialization and update
- then select "OK" two times
Make the Qsys project
- Open Qsys/Platform designer:
- With Quartus 17: Tools->Qsys
- With Quartus 19.1: Tools->Platform designer
- Delete the component clk_0 then save the Qsys project as : qsys_tfht.qsys
- Click on Finish and generate the component with all options left by default
- Under Quartus project add the Qsys project (don't forget to click on add after selected here).
- Right click on qsys_tfht.qsys and set as Top-Level Entity
- Relaunch Qsys by double-clicking on the file in "Project Navigator"
Adding the PCIe and CvP
- In IP Catalog select the component:
Library Interface Protocols PCI Express Avalon-MM Cyclone V Hard IP for PCI Express Avalon-MM Cyclone V Hard IP for PCI Express Intel FPGA IP <- 19.1
- "Add" it to design.
- PCIe configuration:
- Number of Lanes: x1
- Reference clock frequency: 125Mhz
- check Use 62.5 MHz application clock
- check Enable configuration via the PCIe link
- BAR0: 32-bit non-prefetchable memory
- BAR1: 32-bit non-prefetchable memory
- BAR2: 32-bit non-prefetchable memory
- Vendor ID: 0x00001172
- Device ID: 0x0000e001
- Class Code: 0x00001300: there is a bug in quartus, the actual class code is here 13
- click on "Finish"
- Export these signals with name in bold (right click in "Hierarchy", then choose "Connections" and "Export as:..."). Exported signals will be accessible on top of module:
- refclk : refclk
- npor: npor
- hip_serial: hip_serial
- connect Rxm_BAR0 on Cra (always right click in "Hierarchy", then "Connections"...)
- Add CvP component reconfig driver, with all default parameters:
Library Interface Protocols PCI Express Altera PCIe Reconfig Driver Altera PCIe Reconfig Driver Intel FPGA IP <- 19.1
- Connect previous IP signals with:
- coreclkout -> reconfig_xcvr_clk
- coreclkout -> pld_clk
- nreset_status -> reconfig_xcvr_rst
- Add CvP component transceiver, with all default parameters:
Library Interface Protocols Transceiver PHY Transceiver Reconfiguration Controller Transceiver Reconfiguration Controller Intel FPGA IP <- 19.1
- Connect previous PCIe IP signals with:
- coreclkout -> mgmt_clk_clk
- nreset_status -> mgmt_rst_reset
- reconfig_driver.reconfig_mgmt -> reconfig_mgmt
- reconfig_driver.reconfig_busy -> reconfig_busy
- pci.reconfig_to_xcvr -> reconfig_to_xcvr
- pci.reconfig_from_xcvr -> reconfig_from_xcvr
- you should end up with the following wiring:
Adding the DDR3
UniPHY controller
- In the IP catalog, select:
Library Memory Interfaces and Controllers Memory Interfaces with UniPHY DDR3 SDRAM Controller with UniPHY DDR3 SDRAM Controller with UniPHY Intel FPGA IP <- 19.1
- Select checkbox Enable Hard External Memory Interface
- In tab PHY Settings configure:
- Speed Grade: 8
- Memory clock frequency: 375Mhz
- PLL reference clock frequency: 62.5Mhz
- Rate on Avalon-MM interface: Full
- supply Voltage: 1.35V DDR3L
- In tab Memory Parameters
- In "Presets" (right of the window) select: JEDEC DDR3-1G6 2GB X8 and apply
- Memory device speed grade: 400Mhz
- Total interface width: 24
- Row address width: 14
- In tab Controller Settings
- checkbox: Generate power-of-2 data bus widths for Qsys or SOPC Builder
- checkbox: Enable Configuration and Status Register Interface (Internal JTAG)
- checkbox: Enable Error Detection and Correction Logic
- Click on Finish
- Connect following signals:
- coreclkout -> pll_ref_clk
- coreclkout -> mp_cmd_clk_0
- coreclkout -> mp_rfifo_clk_0
- coreclkout -> mp_wfifo_clk_0
- coreclkout -> csr_clk
- nreset_status -> global_reset
- nreset_status -> soft_reset
- nreset_status -> mp_cmd_reset_n_0
- nreset_status -> mp_rfifo_reset_n_0
- nreset_status -> mp_wfifo_reset_n_0
- nreset_status -> csr_reset_n
Span extender
DDR3 memory is to large for PCIe BAR domain. Then we have to use a span extender to adapt PCIe BAR domain to DDR3 domain.
- Select span expander in IP Library:
Library Basic Functions Bridges and Adaptors Memory Mapped Address Span Extender
- Configure it with these options:
- Datapath Width: 64bits
- Slave Word Address Width: 20bits
- press "Finish"
- Connect following PCIe IP signals to extender one:
- coreclckout -> clock
- nreset_status -> reset
- Rxm_BAR0 -> cntl
- Change base address of cntl to 0x0000_4000
- Rxm_BAR1 -> windowed_slave
- And following extender signals to DDR3 one:
- expanded_master -> mem_if_ddr3_emif_0.avl_0
Adding the button and LED (PIO)
Documentation of the PIO component can be found here.
- In IP Library select:
Library Processors and Peripherals Peripherals PIO (Parallel I/O) PIO (Parallel I/O) Intel FPGA IP <- 19.1
- Configure it with these options:
- Width: 2
- Direction: Bidir
- checkbox Synchronously capture
- Edge Type: RISING
- checkbox Generate IRQ
- IRQ Type: EDGE
- click on "Finish"
- export pio.external_connection to pio
- Connect following signals:
- coreclkout -> pio_0.clk
- nreset_status -> pio_0.reset
- Rxm_BAR2 -> pio_0.s1
- pio_0.irq -> Rxmirq
Qsys component schematic
The resulting schematics should give the figure below.
Now click on finish and generate the component with all default parameters kept.
Generate bitstream with Quartus
- Make a first synthesis, in Task window, right click on Analysis & Synthesis then choose Start
Compile Design + Analysis & Synthesis
Note: If you have a synthesis error here with 19.1 be sure that you are using your perl distribution. |
Pinout
- PCIe
- Outside Quartus, create a file named 'pcie_pinout.tcl and edit it with your favorite text editor.
- Copy all lines from here.
- Save your file, then add it to the project inside Quartus
- Run the script with following menu, select it then run:
Tools TCL Scripts ...
- DDR3
- Create a file named ddr3_pinout.tcl and edit it.
- Copy all lines from here (Pinout placement and Technology pinout).
- Save then add the file to project
- Run the script with following menu, select it then run:
Tools TCL Scripts ...
- Button & Led
- Open the pinplanner by double-click in tasks:
Compile Design Analysis & Synthesis I/O Assignment Analysis Pin Planner
- For button pio_export[0] select Location column and set it to : PIN_L9
- For led pio_export[1] select Location column and set it to : PIN_M10
- Close the Pin Planner
Synthesis, Place & route
- Right click on Analysis and Synthesis then Start
- Open a Linux console terminal then go to your project directory and type this:
$ cd ~/tmp/apf6_the_full_howto $ find . -name "*_p0_pin_map.tcl" | xargs sed -iolt "/_mem_if_ddr3_emif_0_p0_get_input_clk_id {/a return \$pll_output_node_id"
Note: This is the DDR3 clock hack, each time synthesis is relaunched, this shell line must be relaunch |
- Right click on Assembler (Generate programming files) then Start
- Convert bitstream to rbf with menu:
File Convert Programming Files...
- set these options:
- Programming file type: Raw Binary File (.rbf)
- Change the name to : output_files/tfht.rbf
- Select SOF Data in Input files to convert then click on Add File...
- add the file apf6_the_full_howto.sof in directory output_files
- checkbox Create CvP files (Generate tfht.periph.rbf and tfht.core.rbf)
- click on Generate
- These files should be present on directory output_files:
ls -l tmp/apf6_the_full_howto/output_files/*.rbf tmp/apf6_the_full_howto/output_files/tfht.core.rbf tmp/apf6_the_full_howto/output_files/tfht.periph.rbf
Note: These bitstreams can be found under armadeus distribution in directory `firmware/tfht/bitstreams` |
Linux BSP configuration
Note: This is not an Armadeus BSP tutorial, the BSP is supposed to be installed and compiled according to this explains |
- Get a compiled BSP with following configuration : apf6_defconfig
- Add PCI debug package with make menuconfig
Target packages ---> Hardware handling ---> [*] pcidebug
$ cp -v output_files/tfht.*.rbf /tftpboot/ ‘output_files/tfht.core.rbf’ -> ‘/tftpboot/tfht.core.rbf’ ‘output_files/tfht.periph.rbf’ -> ‘/tftpboot/tfht.periph.rbf’
Note: Refer to PCIe FPGA loading to understand FPGA configuration with PCIe |
- In U-Boot download the periph.rbf and configure FPGA (don't forget to configure your tftp server ip):
BIOS> tftpboot ${loadaddr} 192.168.0.214:tfht.periph.rbf BIOS> fpga load 0 ${loadaddr} ${filesize}
- Boot Linux
BIOS> boot ... apf6 login: root
- The three BARx should be seen with lspci command :
# lspci -v -s 01:00.0 01:00.0 Unclassified device [0013]: Altera Corporation Device e001 (rev 01) Flags: fast devsel, IRQ 341 Memory at 01800000 (32-bit, non-prefetchable) [disabled] [size=32K] Memory at 01000000 (32-bit, non-prefetchable) [disabled] [size=8M] Memory at 01808000 (32-bit, non-prefetchable) [disabled] [size=32] Capabilities: [50] MSI: Enable- Count=1/4 Maskable- 64bit+ Capabilities: [78] Power Management version 3 Capabilities: [80] Express Endpoint, MSI 00 Capabilities: [100] Virtual Channel Capabilities: [200] Vendor Specific Information: ID=1172 Rev=0 Len=044 <?>
- Configure network:
$ udhcpc
- Download core:
$ tftp -g -r tfht.core.rbf 192.168.0.214
- Load the core bitstream:
$ load_fpga tfht.core.rbf Altera CvP 0000:01:00.0: enabling device (0140 -> 0142) Altera CvP 0000:01:00.0: Found and enabled PCI device with VID 0x1172, DID 0xE001 successfully init Altera CVP with major 248 Altera CvP 0000:01:00.0: Now starting CvP... Altera CvP 0000:01:00.0: CvP successful, application layer now ready 8212+1 records in 8212+1 records out
User access to FPGA
DDR3
- DDR3 is accessible via the BAR1, connect to BAR1:
$ pci_debug -s 01:00.0 -b1
- Read first values of DDR3:
PCI> d32 0 20 00000000: AAAAAAAA AAAAAAAA AAAAAAAA AAAAAAAA 00000010: 55555555 55555555 55555555 55555555
- Write values, then read back:
PCI> c32 0 CAFEDECA PCI> c32 8 CACABEBE PCI> d32 0 20 00000000: CAFEDECA AAAAAAAA CACABEBE AAAAAAAA 00000010: 55555555 55555555 55555555 55555555
- Exit pci_debug
PCI> quit
Only first 8MB (0x800000) are accessible for the moment, to access next 8MB, the span extender must be configured on BAR0.
- Access BAR0 and configure span extender:
$ pci_debug -s 01:00.0 -b0
- Read the span extender windows value at address 0x4000:
PCI> d32 4000 1 00004000: 00000000
- Write the new required value :
PCI> c32 4000 800000 PCI> d32 4000 1 00004000: 00800000
- Quit:
PCI> quit
- then reconnect pci_debug to change for bar1:
$ pci_debug -s 01:00.0 -b1
- Read value :
PCI> d32 0 20 00000000: 6B2171EA DFA5DA76 6CFB3F47 7AF9F526 00000010: DBD85BAB 7F56CF7A 78FAD73B ECF997FE
- Write some values:
PCI> c32 0 12345678 PCI> c32 4 9ABCDEF0 PCI> c32 10 12345678 PCI> c32 14 9ABCDEF0 PCI> d32 0 20 00000000: 12345678 9ABCDEF0 6CFB3F47 7AF9F526 00000010: 12345678 9ABCDEF0 78FAD73B ECF997FE
- Etc...
PIO (Button and led)
As seen on the first schematic of this tutorial, the PIO is plugged on the BAR2 of the PCIe.
- Then first, connect to BAR2 with pci_debug:
$ pci_debug -s 01:00.0 -b2
Button
- Read all PIO registers:
PCI> d32 0 8 00000000: 00000001 00000000
- Leave button pressed and read again:
PCI> d32 0 8 00000000: 00000000 00000000
led
- set PIO[1] (led) as output:
PCI> c32 4 2
- Turn on led
PCI> c32 0 2
- Turn off led:
PCI> c32 0 0
Conclusion
This tutorial should give you the basics to use the CycloneV soldered on the APF6_SP module. Now you should know:
- PCIe:
- How to instanciate the CycloneV hard IP
- How to read/write data with it from Linux with simple access
- DDR3:
- How to instanciate and configure the UniPHY hard IP
- How to connect it to PCIe
- How to read/write simple data
- Simple I/O:
- How to instanciate a simple port for I/O under Qsys
- How to drive it from Linux
Some subjects are not covered by this tutorial:
- Managing PCIe interrupts
- Write a Linux driver
- Increase data bandwidth with PCIe-DMA (up to 140MB/s)
- From the FPGA to the i.MX6
- From the i.MX6 to the FPGA
- Create a custom Qsys component
- Using Serializer/Deserializer
- And more ...
If you are interested in a specific subject, please contact Armadeus System they should do somethings for you.