Flib

From ESL
Jump to navigation Jump to search
WARNING: This library is still under developmentment and not yet intended for public use!


{{#if: Luigi Genovese, CEA Grenoble

Damien Caliste, CEA Grenoble |

Source authors:
Luigi Genovese, CEA Grenoble
Damien Caliste, CEA Grenoble

}}{{#if: LGPL |

License: LGPL

}}{{#if: Project page |

Download: Project page

}}{{#if: |

Documentation: {{{documentation}}}

}}{{#if:
  • Dictionaries
  • Error handling
  • Input file parser
  • Memory management
  • Timing
  • |

    Links to other ESL entries

    | {{#if: |

    Links to other ESL entries

    | {{#if: |

    Links to other ESL entries

    | {{#if: |

    Links to other ESL entries

    |{{#if:
  • YAML specifications
  • |

    Links to other ESL entries

    | {{#if: |

    Links to other ESL entries

    |}}}}}}}}}}}}{{#if:
  • Dictionaries
  • Error handling
  • Input file parser
  • Memory management
  • Timing
  • |

    Functionalities:

    }}{{#if: |

    Algorithms:

      {{{algorithms}}}

    }}{{#if: |

    Generic interfaces:

      {{{generic interfaces}}}

    }}{{#if: |

    APIs:

      {{{apis}}}

    }}{{#if:
  • YAML specifications
  • |

    Data standards:

    }}{{#if: |

    Software:

      {{{software}}}

    }}

    Flib is a library which provides low level functionalities for:

    The library is mainly written in Fortran, but also includes some code written in C.

    Compilation

    ...

    Usage

    At the start of the program the subroutine f_lib_initialize() must be called. Similarly at the end of the program the subroutine f_lib_finalize() must be called. In between these two calls all the functions of f_lib can be used. See the dedicated pages references at the beginning of this page.

    Dictionaries

    flib provides an object called dictionary which is -- strictly speaking -- more than just a dictionaries. It is polymorphic and can be a list or a dictionary, as in the python language. The other difference is that it keeps the order of the elements, which is very useful if one wants to dump its contents to the yaml output.

    These dictionaries are also used in the other parts of the flib library and are thus essential for its proper use. There are many examples in the file dicts.f90.

    Basic routines

    The basis routines provided by flib are

    • dict_init(d) initialize dictionary d?
    • dict_new() start a new dictionary?
    • dict_set(d//'key',val) add the key key to the dictionary d and assign it the value val
    • dict_add(d,val) add the value val to the dictionary d?
    • yaml_dict_dump(d) output the data of dictionary d in the yaml format
    • dict_prepend(d1,d2) add the dictionary d1 before d2?
    • dict_next(d) point to the next dictionary following d?
    • dict_iter(d) start an iterator on the dictionary d (i.e. point to its child)
    • dict_next(d) point to the next dictionary?
    • list_new(ds) create a list from a table of dictionaries
    • dict_free(d) destroy the dictionary d

    Some routines require the YAML output of flib.

    Minimal example

    Here is a minimal example which create a new dictionary d and assigns it the key toto which gets the value 1. Then this value is copied to the variable v, and the dictionary is destroyed:

     use dictionary
     type(dictionary), pointer :: d
     d=>dict_new()
     call set(d//'toto',1)
     v = d//'toto'
     call dict_free(d)
    

    The corresponding example in python would be:

     d = dict()
     d['toto'] = 1
     v = d['toto']
     del(d)
    

    Do dump a dictionary to the yaml format one can simply use:

     use yaml_output
     call yaml_dict_dump(d)
    

    Here is a more complete example. Together with the comments it should be self-explanatory:

     subroutine test_dictionaries()
       use yaml_output                                                     !contains the routines for the yaml output
       use dictionaries                                                    !contains the dictionary routines
       implicit none
       type(dictionary),pointer :: d1, d2, d3
    
       call dict_init(d1)                                                  !initialize the dictionary ''d1''
       call set(d1//'toto',1)                                              !add the key ''toto'' to it and assign it the value 1
       call set(d1//'titi',1.d0)                                           !add the key ''titi'' to it and assign it the value 1.d0
       call set(d1//'tutu',(/ '1', '2' /))                                 !add the key ''tutu'' to it and assign it the array [1,2]
    
       call dict_init(d2)                                                  !initialize the array dictionary ''d2''
       call set(d2//'a',0)                                                 !add the key ''a'' and assign it the value 0
       call set(d1//'List',list_new((/.item.d2, .item.'4', .item.'1.0'/))) !create a list from ''d2'' and the values 4 and 1.0
    
       call yaml_dict_dump(d1)                                             !output the content of ''d1'' in the yaml format
    
       d3 => d1//'New key'                                                 !point to ''d1''?
       call set(d3//'Example',4)                                           !add the key ''Example'' to ''d3'' and assign it the value 4
       call yaml_dict_dump(d3)                                             !output the content of ''d2'' in the yaml format
    
       call yaml_map('List length',dict_len(d1//'List'))                   !print the length of the key ''List'' in dictionary ''d1''
       call yaml_map('Dictionary size',dict_size(d1))                      !print the size of the dictionary ''d1''
       call dict_free(d1)                                                  !destroy the dictionary ''d1''
      
     end subroutine test_dictionaries
    

    This will create the following yaml output:

      toto                                  : 1
      titi                                  : 1.0
      tutu: [1, 2]
      List: [a: 0, 4, 1.0]
      Example                               : 4
      List length                           :  3
      Dictionary size                       :  5
    


    It is also possible to define an iterator from a dictionary. The order is preserved:

     !perform an iterator on dictA
     type(dictionary), pointer :: dictA,dict_tmp
     dict_tmp=>dict_iter(dictA)
     do while(associated(dict_tmp))
         call yaml_map('Iterating in dictA',.true.)
         call yaml_map('Key of dictA',dict_key(dict_tmp))
         call yaml_map('Value of dictA',dict_value(dict_tmp))
         dict_tmp=>dict_next(dict_tmp)
     end do
    

    In this example, the functions dict_key and dict_value are used to get the key and the value, respectively.

    Error handling

    The error handling consists of two steps. First, one has to define the error and the action that has to be taken if this error happens, and in a second step on actually has to trigger the error.

    To define an error on can use the routine f_err_define as follows:

     use dictionary
     call f_err_define(err_name='ERROR_ONE', err_msg='This is an error',err_id=error_one)
    

    The argument err_id is optional and is the way of having the id of the associated error. The module dictionary contains the dictionary type and also the error handling routines.

    If this definition is done, one can raise an exception using f_err_throw as:

     call f_err_throw('We raise the error ERROR_ONE',err_name='ERROR_ONE')
    

    or, if the variable err_one does exist in the same scope, as

     call f_err_throw('We raise the error ERROR_ONE',err_id=error_one)
    

    It is also possible to test a condition and raise the error only if this condition is true. This can be done using the function f_err_raise:

     if (f_err_raise(x>0,'X has to be strictly positive',err_name='ERROR_ONE'))  return
    

    Minimal example

    Here is a small example which shows some basic features of the error handling module. Together with the comments it should be self-explanatory. Some more examples can be found in the file errs.f90

     subroutine test_error_handling()
       use yaml_output                                                          !required to write data in the YAML format
       use dictionaries                                                         !contains the error handling routines
       implicit none
       integer :: ERR_TEST
       external :: abort_test
    
       call yaml_comment('Error Handling Module Test',hfill='~')                !just a comment
    
    
       call f_err_define(err_name='ERR_TEST',&                                  !define the error
            err_msg='This is the error message for the error "ERR_TEST" and'//&
            ' it is written extensively on purpose to see whether the yaml'//&
            ' module can still handle it',&
            err_action='For this error, contact the routine developer',&
            err_id=ERR_TEST,callback=abort_test)
    
       call yaml_map("Raising the TEST error, errcode",ERR_TEST)                !print that the error will now be triggered
       if (f_err_raise(.true.,'Extra message added',err_id=ERR_TEST)) return    !raise the error and return
    
     end subroutine test_error_handling
    
     subroutine abort_test()                                                    !routine that will be called when the error is raised
       use yaml_output
       implicit none
       call yaml_comment('printing error informations',hfill='_')               !just a comment indicating that the error informations will now be written
       call f_dump_last_error()                                                 !print the error information
     end subroutine abort_test
    

    This will produce the output

      #~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Error Handling Module Test
     Raising the TEST error, errcode       :  22
      #_____________________________________________________________ printing error informations
     ERR_TEST:
       Id                                  : 22
       Message:
         This is the error message for the error "ERR_TEST" and it is written extensively on
         purpose to see whether the yaml module can still handle it
       Action                              : For this error, contact the routine developer
       Callback Procedure Address          : 4277190
     Additional Info                       : Extra message added
    


    Memory Management

    In order to use the memory management functionalities one has to include the statement use dynamic_memory in the header of each routine whose allocations and deallocations shoule be traced. This gives access to all public routines.

    Short example showing the use of the wrappers

    A minimal example showing the basic wrappers and their usage is shown below. More examples can be found in the file dynmem.f90

     use dynamic_memory                                     !the module that contains the routines
     real(kind=8),dimension(:,:), allocatable :: ab, weight
     integer,dimension(:,:,:),allocatable :: weight
     integer,dimension(:), pointer :: i1_ptr
    
     call f_routine(id='Routine a')                         !give the name of the routine which does the allocations / deallocations
    
     ab = f_malloc((/ 10, 10 /),id='ab')                    !use the wrapper to allocate the array ab with sizes (1..10,1..10)
     weight=f_malloc((/1.to.8,1.to.8,2.to.4/),id='weight')  !use the wrapper to allocate the array weight woth sizes (1..8,1..8,2..4)
     i1_ptr=f_malloc_ptr(34,id='i1_ptr')                    !use the wrapper to allocate the pointer i1_ptr with size (1..34)
    
     i1_ptr=(/(i+789,i=1,size(i1_ptr))/)                    !shift the lower and upper bounds of i1_ptr?
    
     call f_free_ptr(i1_ptr)                                !use the wrapper to deallocate i1_ptr
     call f_free(weight)                                    !use the wrapper to deallocate weight                      
     call f_free(ab)                                        !use the wrapper to deallocate ab
    
     call f_release_routine()                               !indicate that the profiling for this routine should end
    

    First of all one has to include the module dynamic_memory. Then, at the beginning and end of each subroutine which should explicitely be traced, the statements call f_routine(id=<routine_name> and call f_release_routine() should be included. If they are missing, the tracing will still be performed, but the routine which will be conected with the alloction will be that one in which these calls have been executed for the last time. When allocating an array or pointer, one has to use the generic fuctions f_malloc and f_malloc_ptr, respectively. To deallocate an array or pointer, one has to use the generic subroutines f_free and f_free_ptr, respectively. If at the beginning of the routine f_routine has been called, it is mandatory to ad a call to f_release_routine() at the end.


    Possible allocation shapes

    There are many possibilities to allocate an array. The following examples should be self-explanatory:

     subroutine test_routine
       use dynamic_memory
       integer,dimension(:),allocatable :: i1, i2, j2
       integer,dimension(:),pointer :: j1
       integer,dimension(:,:,:),pointer :: i3
       real(kind=8),dimension(:),allocatable :: d1
       real(kind=8),dimension(:,:),allocatable :: d2
       real(kind=8),dimension(:,:,:),allocatable :: d3
       integer,parameter :: strlen=256
       character(len=strlen),dimension(:),allocatable :: c1
    
       call f_routine(id='test_routine')
    
       d1 = f_malloc(17,id='d1')                !allocate array d1 with size (1..17)
       d2 = f_malloc((/15,2/),id='d2')          !allocate array d2 with size (1..15,1..2)
       d3 = f_malloc([10,3,2],id='d3')          !allocate array d3 with size (1..10,1..3,1..2)
       i2 = f_malloc((/0.to.5,-1.to.1/),id='i2') !allocate array i1 with size (0..5,-1..1)
       j2 = f_malloc(src=i2,id='j2')            !allocate array j2 with the same size as i2
       c1 = f_malloc_str(strlen,5,id='c1')      !allocate array c1 with size (1..5) (note that you have to pass as well strlen)
       i1 = f_malloc0(10,id='i1')               !allocate array i1 with size (1..10) and set to zero
       j1 = f_malloc_ptr(-5.to.5,id='j1')       !allocate pointer j1 to size (-5..5)
       j3 = f_malloc0_ptr((/4,4,2/),id='i3')    !allocate pointer j3 with size (1..4,1..4,1..2) and set to zero
    
       call f_free(d1)                          !deallocate d1
       call f_free(d2)                          !deallocate d2
       call f_free(d3)                          !deallocate d2
       call f_free(i2)                          !deallocate i2
       call f_free(j2)                          !deallocate j2
       call f_free(c1)                          !deallocate c1
       call f_free(i1)                          !deallocate i1
       call f_free_ptr(j1)                      !deallocate j1
       call f_free_ptr(j3)                      !deallocate j3
    
       call f_release_routine()
     end subroutine test_routine
    


    Timing

    Flib can perform a detailed timing by categories or by subroutines. The analysis is done separately for each MPI task and can thus also provide information about load balancing problems.

    Timing by categories

    Here one can manually decide which part of the code should be assigned to which category. It is also possible to split one single subroutine into different categories (e.g. computation ad communication). However it is not possible to assign a given task to several categories, i.e. only one category can be active. Otherwise the code will crash with an error.

    Timing by subroutines

    Flib can perform a timing by routine. To time a subroutine it is enough that this subroutine contains calls to f_routine and f_release_routine (which have been introduced for the Memory management). At the end s detailed tree with the partial times will be created. This mechanism can most easily be shown by a small example. Assume that there are four routines sub1, sub2, sub3 and waste_time:

     subroutine sub1()
       use dynamic_memory
       call f_routine('sub1')
       call sub2()
       call sub2()
       call sub3()
       call f_release_routine()
     end subroutine sub1
    
     subroutine sub2()
       use dynamic_memory
       call f_routine('sub2')
       call waste_time()
       call f_release_routine()
     end subroutine sub2
    
     subroutine sub3()
       use dynamic_memory
       call f_routine('sub3')
       call waste_time()
       call f_release_routine()
     end subroutine sub3
    
     subroutine waste_time()
       implicit none
       integer :: i
       real(kind=8) :: tt
       tt=0.d0
       do i=1,1000000
           tt = tt + sin(real(i,kind=8))
       end do
     end subroutine waste_time
    

    If sub1 is directly called from the main program, this will lead to the following timing output:

     - Main program: [ 0.158,  1, ~*]
       Subroutines:
       - sub1: [ 0.157,  1,  99.64%]
         Subroutines:
         - sub2: [ 0.105,  2,  66.89%]
         - sub3: [ 5.220E-02,  1,  33.25%]
    

    The first entry is the absolut time spent in this routine, the second one the number of calls, and the last one the percentage of time that the routine took with repect to the next upper one (example: sub2 took 66% of the time of sub1 (from where it was called)). Note that the routine waste_time does not appear in the timinig list, as it does not call f_routine.


    YAML I/O

    The Flib library provides some generic routines which allow to output data to the YAML [1] format. The data can be output either from a dictionary or directly by passing it to the routines.

    In order to use the YAML I/O one has to include the statement use yaml_output in the header of each routine which wants to perform YAML output. This gives access to all public routines.

    Minimal example showing the usage of the YAML routines

    The usage of the yaml routines is most easily shown by a small example. Together with the comments it should be self-explanatory.

       subroutine test_yaml()
         use yaml_output
    
         call yaml_comment('Yaml Invoice Example',hfill='-')  !print a comment, starting with a comment sign (#)
    
         call yaml_map('invoice',34843)                       !print an integer in the form "key:value"
         call yaml_map('date',trim(yaml_date_toa()))          !convert the date to a string and adds in in the form "key:value"
    
         call yaml_mapping_open('bill-to',label='id001')      !open a map (i.e. the following entries will be indented) and add a label
    
           call yaml_map('given','Chris')                     !print a string in the form "key:value"
    
           call yaml_mapping_open('address')                  !open a map (i.e. the following entries will be indented)
    
             call yaml_mapping_open('lines')                  !open a map (i.e. the following entries will be indented)                  
    
               call yaml_scalar('458 Walkman Dr.')            !print a single string
               call yaml_scalar('Suite #292')                 !print a single string
    
             call yaml_mapping_close()                        !close the innermost map
    
           call yaml_mapping_close()                          !close the next map
    
         call yaml_mapping_close()                            !close the outermost map
    
         call yaml_map('ship_to','*id001')                    !print a string in the form "key:value"
    
         call yaml_sequence_open('product')                   !open a sequence, i.e. a list of elements
    
           call yaml_sequence(advance='no')                   !add an element to the list
    
           call yaml_map('sku','BL394D')                      !print a string in the form "key:value"
           call yaml_map('quantity',4)                        !print an integer in the form "key:value"
           call yaml_map('description','Basketball')          !print a string in the form "key:value"
           call yaml_map('price',450.,fmt='(f6.2)')           !print a real number with a specified format in the form "key:value"
           call yaml_map('parcel dimensions',(/30,32,35/))    !print an array of integers in the form "key:value"
    
           call yaml_sequence(advance='no')                   !add an element to the list
    
           call yaml_mapping_open(flow=.true.)                !open a map where the entries are on the same line
    
             call yaml_map('sku','BL4438H')                   !print a string in the form "key:value"
             call yaml_map('quantity',1)                      !print an integer in the form "key:value"
             call yaml_newline()                              !start a new line
             call yaml_map('description','Super Hoop')        !print a string in the form "key:value"
             call yaml_map('price',2392.,fmt='(f8.2)')        !print a real number with a specified format in the form "key:value"
             call yaml_map('parcel dimensions',(/120,20,15/)) !print an array of integers in the form "key:value"
    
           call yaml_mapping_close()                          !close the current map
    
         call yaml_sequence_close()                           !close the list
    
         call yaml_map('tax',251.42,fmt='(f6.2)')             !print a real number with a specified format in the form "key:value"
         call yaml_map('total',4443.52d0,fmt='(f6.2)')        !print a real number with a specified format (which is wrong on purpose) in the form "key:value"
         call yaml_map('comments','Late afternoon is best. Backup contact is Nancy Billsmer @ 338-4338.')  !print a comment, starting with a comment sign (#)                                 
    
       end subroutine test_yaml
    


    This will produce the output

      #-------------------------------------------------------------------- Yaml Invoice Example
     invoice                               :  34843
     date                                  :  2014-07-04
     bill-to: &id001
       given                               : Chris
       address:
         lines:
           458 Walkman Dr.
           Suite #292
     ship_to                               : *id001
     product:
     - sku                                 : BL394D
       quantity                            :  4
       description                         : Basketball
       price                               :  450.00
       parcel dimensions                   :  [  30,  32,  35 ]
     -  {sku: BL4438H, quantity:  1, 
     description: Super Hoop, price:  2392.00, parcel dimensions:  [  120,  20,  15 ]}
     tax                                   :  251.42
     total                                 :  4443.520000000000
     comments:
       Late afternoon is best. Backup contact is Nancy Billsmer @ 338-4338.
    

    YAML input

    The input of data in the YAML format is not yet as mature as the output. In particular it is not yet a standalone functionality of Flib. In addition it requires some knowledge about the dictionaries of Flib (these are as well used for the output, but are more or less completely hidden from the user).

    To read an input file in the YAML format one has to use the following routine which reads the file and adds its content to the dictionary.

     subroutine read_input_dict_from_files(radical,mpi_env,dict)
    

    To extract the input parameters from the dictionaries one has to use the routine

     subroutine inputs_from_dict(in, atoms, dict)
    

    Dependencies

    Although Flib is a fortran library, it also has some parts written in C and thus requires a C compiler.

    Furthermore a recent version of LibYAML is required.

    Parallelization

    Flib is a low level library and exhibits therefore no internal parallelization.

    Current developments

    • The low level routines from BigDFT (including memory and time management, error handling and YAML parsing) are being separated out into a standalone library. (Stephan Mohr, Luigi Genovese, and Damien Caliste).