The HOMER address is colliding with an NVDIMM address and should not

When having NVDIMMs, the HOMER address needs to be adjusted to avoid
running into the NVDIMM's address.

Currently the method mem_utils.C::get_top_homer_mem_addr() makes a call
to the method mem_utils.C::get_top_mem_addr().  The get_top_mem_addr()
is the method that actually gets the HOMER address.  Once the HOMER
address has been retrieved, then the method call to
NVDIMM::get_top_addr_with_no_nvdimms() will verify that the
returned HOMER address does not collide with an NVDIMM address and if so
the HOMER address is adjusted before returning to the user.

The logic looks like this:
mem_utils.C::get_top_homer_mem_addr()
                           -> mem_utils.C::get_top_mem_addr()
                           -> NVDIMM::get_top_addr_with_no_nvdimms()

The problem is, in some places in the code, to get the top address,
the call to get_top_mem_addr() is made directly but the returned value
is never checked against and NVDIMM.  Changing the logic to be:
mem_utils.C::get_top_homer_mem_addr() ->
        mem_utils.C::get_top_mem_addr() ->
                NVDIMM::get_top_addr_with_no_nvdimms()

Where the check against an NVDIMM is done, at the end of the call to the
method get_top_mem_addr(), and resolves any NVDIMM collisions.  Now any
call to get_top_homer_mem_addr() and/or get_top_mem_addr(), both should
and will return the same value, and that value compensates for an
NVDIMM.

The code that deals with SMF being enabled, well, there should be no NVDIMMs
so an assertion is added to ensure that this will be the case.

Change-Id: Iab5885da261b7b9d3c5172fea69b6b85b3da45d8
CQ: SW484250
Reviewed-on: http://rchgit01.rchland.ibm.com/gerrit1/91926
Reviewed-by: Daniel M Crowell <dcrowell@us.ibm.com>
Tested-by: Jenkins Server <pfd-jenkins+hostboot@us.ibm.com>
Reviewed-by: Ilya Smirnov <ismirno@us.ibm.com>
Tested-by: Jenkins OP Build CI <op-jenkins+hostboot@us.ibm.com>
Tested-by: Jenkins OP HW <op-hw-jenkins+hostboot@us.ibm.com>
Tested-by: FSP CI Jenkins <fsp-CI-jenkins+hostboot@us.ibm.com>
Reviewed-by: Nicholas E Bofferding <bofferdn@us.ibm.com>

src/usr/isteps/mem_utils.C

@@ -5,7 +5,7 @@
 /*                                                                        */
 /* OpenPOWER HostBoot Project                                             */
 /*                                                                        */
-/* Contributors Listed Below - COPYRIGHT 2018,2019                        */
+/* Contributors Listed Below - COPYRIGHT 2018,2020                        */
 /* [+] International Business Machines Corp.                              */
 /*                                                                        */
 /*                                                                        */
@@ -54,6 +54,10 @@ uint64_t get_top_mem_addr()
 
     } while(0);
 
+    // Need to make sure we don't choose a range that is owned by
+    // the NVDIMMs
+    l_top_addr = NVDIMM::get_top_addr_with_no_nvdimms(l_top_addr);
+
     return l_top_addr;
 }
 
@@ -161,6 +165,14 @@ uint64_t get_top_smf_mem_addr()
         l_top_addr = std::max(l_top_addr, get_top_smf_mem_addr(l_pProc));
     }
 
+    // There should be no NVDIMM's while in SMF mode, but there is no fast and
+    // easy way to determine if the system has NVDIMMs or not.  This call should
+    // return the same input value, if there are no NVDIMMs.  If the value
+    // returned is not the same as the input value, then there is a
+    // configuration issue, therefore assert stating so.
+    assert((l_top_addr == NVDIMM::get_top_addr_with_no_nvdimms(l_top_addr)),
+           "Sytem Configuration Issue: Found NVDIMMs while in SMF mode.");
+
     return l_top_addr;
 }
 
@@ -187,13 +199,20 @@ uint64_t get_top_smf_mem_addr(const TARGETING::Target* i_proc)
 }
 
 /**
- * @brief Utility function to fetch the top of the HOMER memory;
- *        when SMF is enabled, the function will return the highest
- *        available SMF BAR on a proc, so the top of the HOMER memory
- *        will be pointed to the top of SMF memory, and all of the
- *        contents will be put in secure memory space. When SMF is not
- *        enabled, the function will return the highest known address
- *        on the system.
+ * @brief Utility function to fetch the top of the HOMER memory address
+ *
+ * @details * When SMF is enabled, the function will return the highest
+ *            available SMF BAR on a proc, so the top of the HOMER memory
+ *            will be pointed to the top of SMF memory, and all of the
+ *            contents will be put in secure memory space.
+ *          * When SMF is not enabled, the function will return the highest
+ *            known address on the system, that does not collide with an
+ *            NVDIMM's address.
+ *
+ * @note: There should be no NVDIMMs when in SMF mode
+ *
+ * @return the top HOMER memory address that does not collide with an NVDIMM
+ *         if there are NVDIMMs.
  */
 uint64_t get_top_homer_mem_addr()
 {
@@ -209,11 +228,6 @@ uint64_t get_top_homer_mem_addr()
             l_top_homer_addr = get_top_mem_addr();
         }
 
-        // Need to make sure we don't choose a range that is owned by
-        //  the NVDIMMs
-        l_top_homer_addr =
-          NVDIMM::get_top_addr_with_no_nvdimms(l_top_homer_addr);
-
     }while(0);
 
     return l_top_homer_addr;