=============== WPS/WRF Install =============== Overview ======== The directory (with respect to the top of the repository) **nwp_install/install-wrf/** is set up to support installation of WRF distributions. Current work has focused specifically on the installation of the WRFv4.3 WPS/WRF packages in a CTBTO-like CentOS 7 environment. However, I have used this kind of approach for many years on many systems for many distributions, so I think it is "generally portable." The components of this directory consist of * **provision.py** - this is a script I used at NCAR a few years ago, and is in here just as a sample to copy ideas from * **wrfv4.0-2022-03-25/**, **wrfv4.3_2021-08-12/**, **wrfv4.3_2022-04-25/** -- these are the subdirectories with all of the installation code, one for each version [**NEED TO EXPLAIN THE DIFFERENCES IN THESE VERSIONS**]. My intent is to package other versions, or significant modifications to existing versions, in much the same way, with descriptive title and date of implementation. In this way, it is expected that users will be able to create their own custom distributions or have a choice of other distributions. * **README-versions.md** - brief description of the WPS/WRF versions that have been successfully implemented and tested. * **test/** -- this subdirectory contains met data, configuration files, and sample output for testing an installed distribution. All of the above are explained in more detail below, focusing on one of the WPS/WRF distributions. ---- wrfv4.3_2021-08-12 ================== This is the set of resources which will install and setup, in user space (if desired) a complete WPS/WRF distribution with a user setup script to facilitate creation of custom domains and simulations, each isolated from each other. * **setup_install_env.sh** - sets up the system-specific environment variables needed for successful installation. Most importantly, this includes paths to libraries and executables needed for WPS/WRF installation. * **wrfinstall.py** - evolving script for installing and testing WRF and WPS (and maybe later, GSI and UPP) in a CTBTO CentOS 7 environment * Fetches (from WRF's github) source codes * Compiles in specified location * Copies in a `wrfusersetup` script that allows users to create unlimited custom modeling domains based on this distribution * Runs a sample WPS/WRF test case to verify things work. * **wrfusersetup/** - directory with Python driver to create a user-executed script for installing custom WRF/WPS run directories based on links and copies of pertinent files from the main WRF/WPS distribution Sample usage ------------ To install in a new dir, the base dir will be created (not recursively) if the path to it is valid. So, for example, if the user specifies installation in directory ``/home/morton/simulations/WRF4.3``, the directory ``/home/morton/simulations/`` must already exist, and the subdirectory ``WRF4.3`` will be created as part of the installation process. Basic help: .. code-block:: bash $ ./wrfinstall.py --help usage: wrfinstall.py [-h] --baseinstalldir BASEINSTALLDIR --geogdatadir GEOGDATADIR [--forceinstall] optional arguments: -h, --help show this help message and exit --baseinstalldir BASEINSTALLDIR, -b BASEINSTALLDIR (Required) Base directory for installed components --geogdatadir GEOGDATADIR, -g GEOGDATADIR (Required) Directory of the WPS geog data --forceinstall (Optional, defaults to False if not specified) Full install over any existing installation The required ``--geogdatadir`` argument specifies a full path to the WPS geog data, and the install program makes a link to it in the new distribution directory. The WPS geog data is discussed in more detail in the **WRF Install Technical Notes** section. The ``--forceinstall`` flag is there to help the user bypass compilation if desired (for example, for development of the *wrfinstallscript* or for performing basic tests on the installation). By default, if ``--forceinstall`` is not specified on the command-line, and the **executables already exist** in the base installation directory, then reinstallation will be skipped. We can think of this as kind of a safety device that keeps us from accidentally removing a good installation and having to go through the full compilation process again. On the other hand, there may be times when the user wants to re-install over an existing installation, with recompilation. In this case, using the ``--forceinstall`` flag will enable the recompilation. So, to perform a basic first installation, the user would first edit ``setup_install_env.sh`` and ensure the paths to libraries, etc. are specified correctly, and would then: .. code-block:: bash $ . setup_install_env.sh $ ./wrfinstall.py --baseinstalldir ~/WRFV4.3-Distribution --geogdatadir /home/ctbtuser/tempgeog . . . 2021-08-13 19:43:26,626 - DEBUG - wrfinstall.py:nam_nodak_test:634 --> metrid files successfully produced 2021-08-13 19:43:26,626 - INFO - wrfinstall.py:nam_nodak_test:636 --> WPS test successfully completed!!! 2021-08-13 19:43:26,626 - DEBUG - wrfinstall.py:nam_nodak_test:645 --> cwd is: /tmp/nam_nodak_2021-08-13_19_43_19/WRF 2021-08-13 19:43:26,626 - DEBUG - wrfinstall.py:nam_nodak_test:654 --> Copied in namelist.input 2021-08-13 19:43:26,626 - DEBUG - wrfinstall.py:nam_nodak_test:659 --> Linking to met_em* : ['ln -sf ../WPS/met_em* .'] 2021-08-13 19:43:26,629 - DEBUG - wrfinstall.py:nam_nodak_test:664 --> Running real.exe: ['/usr/lib64/openmpi/bin/mpirun', '-np', '1', './real.exe'] starting wrf task 0 of 1 2021-08-13 19:43:27,249 - DEBUG - wrfinstall.py:nam_nodak_test:671 --> real.exe files successfully produced 2021-08-13 19:43:27,249 - DEBUG - wrfinstall.py:nam_nodak_test:676 --> Running wrf.exe: ['/usr/lib64/openmpi/bin/mpirun', '-np', '1', './wrf.exe'] starting wrf task 0 of 1 2021-08-13 19:44:08,068 - DEBUG - wrfinstall.py:nam_nodak_test:683 --> wrfout files successfully produced 2021-08-13 19:44:08,068 - INFO - wrfinstall.py:nam_nodak_test:685 --> Apparent successful completion of test... and it may take up to 10-30 minutes for compilation depending on environment. If I try it again, it will bypass the existing installation of WPS and WRF (which is where all the compile time is spent), since, by default, the ``--forceinstall`` option is set to False). .. code-block:: bash $ ./wrfinstall.py --baseinstalldir ~/WRFV4.3-Distribution --geogdatadir /home/ctbtuser/tempgeog 2021-08-13 19:45:11,389 - INFO - wrfinstall.py:main:56 --> Starting install... 2021-08-13 19:45:11,389 - DEBUG - wrfinstall.py:main:57 --> cli_args: Namespace(baseinstalldir='/home/ctbtuser/WRFV4.3-Distribution', forceinstall=False) 2021-08-13 19:45:11,389 - DEBUG - wrfinstall.py:main:67 --> MPIF90: /usr/lib64/openmpi/bin/mpif90 2021-08-13 19:45:11,389 - DEBUG - wrfinstall.py:main:74 --> MPICC: /usr/lib64/openmpi/bin/mpicc 2021-08-13 19:45:11,389 - DEBUG - wrfinstall.py:main:82 --> MPIRUN: /usr/lib64/openmpi/bin/mpirun 2021-08-13 19:45:11,389 - DEBUG - wrfinstall.py:main:92 --> NETCDF_MOD_INC: /usr/lib64/gfortran/modules 2021-08-13 19:45:11,389 - INFO - wrfinstall.py:main:134 --> /home/ctbtuser/WRFV4.3-Distribution exists... moving ahead... 2021-08-13 19:45:11,389 - INFO - wrfinstall.py:main:138 --> Starting WRF install in: /home/ctbtuser/WRFV4.3-Distribution 2021-08-13 19:45:11,389 - DEBUG - wrfinstall.py:wrf_install:195 --> wrf_install_dir: /home/ctbtuser/WRFV4.3-Distribution/WRF 2021-08-13 19:45:11,389 - INFO - wrfinstall.py:wrf_install:209 --> WRF executables exist and force_install is set to False. Skipping ... 2021-08-13 19:45:11,389 - INFO - wrfinstall.py:main:149 --> Starting WPS install in: /home/ctbtuser/WRFV4.3-Distribution 2021-08-13 19:45:11,389 - DEBUG - wrfinstall.py:wps_install:339 --> wps_install_dir: /home/ctbtuser/WRFV4.3-Distribution/WPS 2021-08-13 19:45:11,389 - INFO - wrfinstall.py:wps_install:355 --> WPS executables exist and force_install is set to False. Skipping ... 2021-08-13 19:45:11,389 - INFO - wrfinstall.py:run_tests:516 --> Running NAM NODAK test 2021-08-13 19:45:11,389 - DEBUG - wrfinstall.py:nam_nodak_test:545 --> Beginning of NAM NODAK test . . . and move on to the copying of ``wrfusersetup`` and then the running of a test case. But, if I include the ``--forceinstall`` flag, then it will recompile WRF and WPS as part of the installation and test process. ---- wrfusersetup program -------------------- The ``wrfusersetup.py`` program is copied into the new distribution directory, and allows the user to create isolated, custom run directories based on this new distribution. The new run directories contain links to the large files and executables of the distribution directory, and copies of other files. It is invoked as in the following example. Assuming that the WRF distribution was installed into ``/home/ctbtuser/WRFV4.3-Distribution``, I can use this program to create a run dir as follows .. code-block:: bash $ /home/ctbtuser/WRFV4.3-Distribution/wrfusersetup.py MyNewCase WRF src distribution directory: /home/ctbtuser/WRFV4.3-Distribution geog data dir: /home/ctbtuser/WRFV4.3-Distribution/GEOG_DATA Creating domain: MyNewCase in directory: /home/ctbtuser Setting up WPS run dir: /home/ctbtuser/MyNewCase/WPS Done creating geogrid items Done creating metgrid items Done creating ungrib items Done creating util items Setting up WRF run dir: /home/ctbtuser/MyNewCase/WRF WRF run dir set up completed and I can verify a full run directory as follows .. code-block:: bash $ tree -L 1 -F MyNewCase MyNewCase ├── GEOG_DATA -> /home/ctbtuser/WRFV4.3-Distribution/GEOG_DATA/ ├── WPS/ └── WRF/ 3 directories, 0 files ---- Summary ======= This set of installation programs has been created to make it somewhat easy to deploy a wide range of versions of WRF/WPS, allowing for great flexibility in modeling environments. The ``wrfusersetup.py`` script allows the user to create a number of custom run directories based on the distribution of their choice. In the past, I've added other WRF components (e.g. GSI, UPP) to this kind of system, and that could be done in the future. Please refer to the :doc:`WRF Install Technical Notes ` section if more detail is needed. .. toctree:: TechNotes/index