Lazy Loading DataTable con IceFaces

20:38Java, SvliluppoStefano

IceFaces fornisce ai programmatori numerosi componenti per facilitare lo sviluppo di applicazioni web.
Tra questi, uno dei piu’ utili e’ sicuramente la ice:dataTable.

Questo componente, unito all’ice:dataPaginator permette infatti di paginare l’intero data set di una tabella, mostrando solamente N righe per pagina.

La debolezza principale di questo componente risiede nella difficolta’ di gestire data set di grandi dimensioni: per funzionare il componente ice:dataTable si aspetta infatti di ricevere una lista contenente tutte le righe che andra’ via via a mostrare.
Finche’ siamo nell’ordine di qualche centinaia di record, non c’e’ nessun problema a fornire al componente l’intera lista di risultati; se questi iniziano ad essere migliaia, tenere in memoria una tale quantita’ di oggetti puo’ essere un problema.

La soluzione consiste nel gestire la lista dei risultati in modo lazy: sara’ la lista stessa a recuperare i record da mostrare nella pagina corrente solo quando ce ne sara’ un effettivo bisogno.

Oltre a questo, sara’ inoltre necessario gestire il paginatore in modo che mostri comunque l’insieme di tutte le pagine corrispondente al numero totale di risultati. Il paginatore calcola infatti il totale delle pagine invocando il metodo size() della lista fornita alla tabella.

Vediamo ora tre implementazioni possibili per applicare il lazy loading alla lista.

Saranno inizializzate nel backing bean e invocate nel template .jsf in questo modo:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
<ice:dataTable id="data"
              value="#{backingBean.lazyLoadingList}"
              rows="10" >
....
</ice:dataTable>
<ice:dataPaginator id="scroll_1"
                  for="data"
                  fastStep="10"
                  pageCountVar="pageCount"
                  pageIndexVar="pageIndex"
                  paginator="true"
                  paginatorMaxPages="9">
...
</ice:dataPaginator>

1. LazyLoadingList

Questa lista mantiene in memoria una lista di oggetti corrispondente alla prima pagina. Quando l’utente navighera’ nelle pagine successive, queste verranno di volta in volta recuperate dal metodo get() e salvate in una diversa lista. Da notare che il costruttore della lista accetta in entrata il parametro totalResultsNumber, cioe’ il numero totale dei risultati da mostrare in tabella. Questo parametro sara’ dunque il risultato di una count query.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
package com.devinterface.lazyloading;

import java.util.AbstractList;
import java.util.List;

/**
 * This list loads and stores only the first page and the current page.
 * If pageSize is equals to totalResultsNumber, the dataTable will be non paginated: the first query will retrieve all the dataset.
 *
 * @param <T>
 */
public class LazyLoadingList<T> extends AbstractList<T>
{
  private IDataProvider<T> dataProvider;

  private List<T> firtsPageData;
  private List<T> currentPageData;

  private int currentPage = -1;
  private int totalResultsNumber;
  private int pageSize;

  /**
   * @param dataProvider, the object that will perform the query
   * @param pageSize, the number of rows to be showed in a table page
   * @param totalResultsNumber, the total number of rows as result of the database count query.
   */
  public LazyLoadingList(IDataProvider<T> dataProvider, int pageSize, int totalResultsNumber)
  {
    this.dataProvider = dataProvider;
    this.totalResultsNumber = totalResultsNumber;
    this.pageSize = pageSize;
  }

  @Override
  public T get(int i)
  {
    if (i < pageSize)
    {
      if (firtsPageData == null)
        firtsPageData = dataProvider.getBufferedData(i, pageSize);
      return firtsPageData.get(i);
    }
    int page = i / pageSize;

    if (page != currentPage)
    {
      currentPage = page;
      currentPageData = dataProvider.getBufferedData(i, pageSize);
    }

    return currentPageData.get(i % pageSize);
  }

  @Override
  public int size()
  {
    return totalResultsNumber;
  }

  public void setTotalResultsNumber(int totalResultsNumber)
  {
    this.totalResultsNumber = totalResultsNumber;
  }

  @Override
  public void clear()
  {
    firtsPageData.clear();
    currentPageData.clear();
  }

}

2. LazyLoadingMapList

Questa lista mantiene in memoria i record che man mano recupera dal database in una HashMap. Da notare che la mappa non viene mai svuotata, quindi nel caso peggiore in cui l’utente scorre una ad una tutte le pagine la mappa conterra’ tutti gli elementi del dataset.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
package com.devinterface.lazyloading;

import java.util.AbstractList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

/**
 * This list stores all records in a map with key = "table row index" and value = "T"
 * If pageSize is equals to totalResultsNumber, the dataTable will be non paginated: the first query will retrieve all the dataset.
 * Note that if a user moves from first to last page, one page at time, all record will be in memory.
 *
 * @param <T>
 */
public class LazyLoadingMapList<T> extends AbstractList<T>
{
  private IDataProvider<T> dataProvider;

  private int totalResultsNumber;
  private int pageSize;

  /** cache of loadedData items */
  private Map<Integer, T> loadedData;

  /**
   * @param dataProvider, the object that will perform the query
   * @param pageSize, the number of rows to be considered as "a page"
   * @param totalResultsNumber, the total number of rows as result of the database count query.
   */
  public LazyLoadingMapList(IDataProvider<T> dataProvider, int pageSize, int totalResultsNumber)
  {
    this.dataProvider = dataProvider;
    this.totalResultsNumber = totalResultsNumber;
    this.pageSize = pageSize;
    loadedData = new HashMap<Integer, T>();
  }

  @Override
  public T get(int i)
  {
    if (!loadedData.containsKey(i))
    {
      int pageIndex = i / pageSize;
      List<T> results = dataProvider.getBufferedData(i, pageSize);
      for (int j = 0; j < results.size(); j++)
      {
        loadedData.put(Integer.valueOf(pageIndex * pageSize + j), (T) results.get(j));
      }
    }
    return loadedData.get(i);

  }

  @Override
  public int size()
  {
    return totalResultsNumber;
  }

  public void setTotalResultsNumber(int totalResultsNumber)
  {
    this.totalResultsNumber = totalResultsNumber;
  }

  @Override
  public void clear()
  {
    loadedData.clear();
  }

}

3. LazyLoadingBufferedMapList

Questa lista rappresenta un’evoluzione della precedente, mantenendo in memoria solo gli elementi corrispondenti alla pagina corrente, alla precedente e alla successiva.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
package com.devinterface.lazyloading;

import java.util.AbstractList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

/**
 * This list stores records in a map with key = "table row index" and value = "element"
 * It keeps a buffer of 3 pages: the first time it will load pages 1,2,3.
 * When user moves to page 4, all stored data will be cleared and will be retrieved page 3,4,5.
 * Basically, this list will store current page, previous page and next page.
 * If bufferSize is equals to totalResultsNumber, the dataTable will be non paginated: the first query will retrieve all the dataset.
 *
 * @param <T>
 */
public class LazyLoadingBufferedMapList<T> extends AbstractList<T>
{
  private IDataProvider<T> dataAdapter;

  private int totalResultsNumber;
  private int pageSize = 10;
  private int bufferSize = 30;

  /** cache of loadedData items */
  private Map<Integer, T> loadedData;

  /**
   *
   * @param dataAdapter, the object that will perform the query
   * @param pageSize, the number of rows to be considered as "a page"
   * @param totalResultsNumber, the total number of rows as result of the database query.
   */
  public LazyLoadingBufferedMapList(IDataProvider<T> dataAdapter, int pageSize, int totalResultsNumber)
  {
    this.dataAdapter = dataAdapter;
    this.totalResultsNumber = totalResultsNumber;
    this.pageSize = pageSize;
    this.bufferSize = pageSize * 3;
    loadedData = new HashMap<Integer, T>();
  }

  @Override
  public T get(int i)
  {
    if (!loadedData.containsKey(i))
    {
      clearMap();

      int startRow = getStartRow(i);

      int numElementToFind = bufferSize;
      if ((startRow + numElementToFind) > totalResultsNumber)
        numElementToFind = totalResultsNumber - startRow;

      List<T> results = dataAdapter.getBufferedData(startRow, numElementToFind);
      for (int j = 0; j < results.size(); j++)
        loadedData.put((startRow + j), (T) results.get(j));
    }
    return loadedData.get(i);

  }

  /**
   * clears the map except the first element that MUST be kept
   */
  private void clearMap()
  {
    T firstElement = loadedData.get(0);
    loadedData.clear();
    loadedData.put(0, firstElement);
  }

  /**
   * Calculates the index of the previous page's first element
   * @param i, the current row index
   * @return the index of the previous page's first element
   */
  private int getStartRow(int i)
  {
    int currentPage = (i / pageSize) + 1;

    int firstIndexOfCurrentPage = pageSize * (currentPage - 1);

    int firstIndexOfPreviusPage = firstIndexOfCurrentPage - (bufferSize / 3);

    if (firstIndexOfPreviusPage < 0)
      firstIndexOfPreviusPage = 0;

    return firstIndexOfPreviusPage;
  }

  @Override
  public int size()
  {
    return totalResultsNumber;
  }

  public void setNumResults(int numResults)
  {
    this.totalResultsNumber = numResults;
  }

  @Override
  public void clear()
  {
    loadedData.clear();
  }
}

Conclusioni

Tutte le implementazioni danno la possibilita’ di recuperare record in modo lazy. Sicuramente la seconda rappresenta la soluzione potenzialmente piu’ debole in quanto nel caso peggiore manterra’ in memoria tutto il dataset.
Sicuramente la terza implementazione rappresenta la soluzione migliore sotto tutti i punti di vista. Permette infatti di avere in memoria solo un numero limitato di record e di soddisfare potenzialmente eventuali “avanti e indietro” dell’utente.

I sorgenti nel mio repository su GitHub, LazyLoadingDataTable

Tags: , , ,


About Stefano

Stefano Mancini is a co-founder of DevInterface.

After graduating in Computer Science, he first specialized in Java/J2EE development by participating in several international projects in the pharmaceutical and banking ambits.

Enthusiast of agile development, like SCRUM for project management and eXtreme Programming for code writing, he then moved to dynamic languages like Ruby and Python.

About DevInterface

We are an information and communication technology agency. Our mission is to provide web application development, design services and communication strategies. We specialize in building web applications with modern and efficient frameworks.

Random Posts

22 Responses to “Lazy Loading DataTable con IceFaces”

  1. Lazy Loading DataTable with IceFaces « XP on Rails scrive:
    14 marzo 2010 alle 20:48

    [...] Original Post [...]

  2. Martin scrive:
    16 marzo 2010 alle 13:04

    works great! thanks!
    i’m trying to extend this with sorting mechanism.. could you give me a hint?

  3. Stefano scrive:
    16 marzo 2010 alle 16:08

    Hi Martin.

    Suppose to have a sortable column.

    The best way to handle sorting is to add an actionListener to the ice:commandSortHeader that clears the list (e.g. calls the clear() method on the lazy list) and forces to load the page again.

  4. Martin scrive:
    17 marzo 2010 alle 08:17

    did it =)

    0}”
    sortColumn=”#{universalRequestBean.sortColumnName}”
    sortAscending=”#{universalRequestBean.ascending}”>

    in the Getter of the lazy list i check changes of the properties ’sortColumnName’ and ‘ascending’. the list reloads data if necessary

  5. Martin scrive:
    17 marzo 2010 alle 08:20

    oh, there happened something to the ice:datatable – Tag in my post ^^

    the important properties here again:

    value=”#{universalRequestBean.lazyFadsList}”
    sortColumn=”#{universalRequestBean.sortColumnName}”
    sortAscending=”#{universalRequestBean.ascending}”

  6. EDH scrive:
    17 maggio 2010 alle 11:44

    Hi,

    I thought I’d post my question here because I’ve got a problem related to datatable.

    Before I turn to the paginator solution you excellently describe I’d like to find out whether I’m not doing something wrong.

    The problem is I’m using a datatable to display around 7000 ISO language codes.

    I’m not using pagination at the moment. It takes a while, as expected, to display the list. I can see that the majority of the work is located in the browser (IE or FF) because I see the processer shooting up to 99% for the IE/FF process.
    I could live with waiting a bit longer to retrieve the list, because it’s big.

    However, sorting is very very slow (5min) and I’m wondering what could be the cause ? Any idea’s why sorting is much more slower than displaying the list for the first time ? Are there any icefaces settings that could resolve this ?

    thanks,
    EDH

  7. Stefano scrive:
    18 maggio 2010 alle 08:44

    Hi EDH.

    When you call a column sort the dataTable will invoke an alphanumeric sorting on the value of selected column.

    This sorting should be handled on server side by calling the compareTo method of your field’s class.

    Then it should display again the page.

    So the total time should be given by:

    - the time to perform all 7 phases of JSF lifecycle (maybe one of them will take too much time)
    - sort the dataset (be sure to not perform the query a second time)
    - render the table again

    I suggest to implement the javax.faces.event.PhaseListener in order to log the total time spent on each phase.

  8. Jorge Ospina scrive:
    17 giugno 2010 alle 19:01

    I need help, I can send the full code. I look at BackingBean and all implementation.
    thanks

  9. radu scrive:
    9 settembre 2010 alle 15:21

    Hello!
    I have a problem with ICEFACES JSF implementation.
    Even if dataProvider.getBufferedData return a page section of records(10) and set correctly totalResultsNumber to (200), the paginator still display count of 10 records and one page. It seems that it does not use size() function for total number, but use an internal count of records.
    Please help with this issue!

  10. cabaji scrive:
    14 settembre 2010 alle 01:28

    Im trying to implement your paginator, but i dont undertand what to do with the IDataProvider->

    its a custom class? with the query?

    where do i put the query to the database, and how i put in the backing bean the lazyloading list???????

  11. Stefano scrive:
    14 settembre 2010 alle 08:46

    @Cabaji: the IDataProvider must be implemented by a your class in order to perform the paginated query.

    You must implement the method

    1
     List<T> getBufferedData(int startRow, int offset);

    that gets as input a start row and the number of row to fetch and returns a list that contains offset elements

  12. cabaji scrive:
    14 settembre 2010 alle 17:37

    Thx works great!, incredible code, but can u explain to me how is the interaction of the paginator with the database?,

  13. cabaji scrive:
    29 settembre 2010 alle 23:47

    Hi i tried your lazy loading paginator…> but when i debugged the application, i noticed, that calls the database a lot of times, when press search to load the database, so im looking that its doing nothing when i click next in the datapaginator. Im doing something wrong?

  14. phonix.cao scrive:
    4 marzo 2011 alle 08:17

    thanks .

  15. KrystianG scrive:
    19 maggio 2011 alle 01:48

    Hi, I’d like to use this code in my school project. Can I do this?

  16. Stefano scrive:
    19 maggio 2011 alle 07:48

    Krystian, of course, you can use this code in your project.

    It could have a MIT License :-)

  17. vijay t scrive:
    6 giugno 2011 alle 10:01

    Hi,
    gr8..piece of code
    help me a lot

    Thanks.

  18. Roberto scrive:
    9 settembre 2011 alle 08:48

    Could you provide a full example code.. JSF pages, BackingBean..etc..
    Thanks!

  19. Will scrive:
    31 agosto 2012 alle 15:25

    Fantastic article!

  20. MinhQuan scrive:
    10 ottobre 2012 alle 05:42

    Dear,
    I from Vn, my english not good, i read your post, but i can’t use it, can you send me full source code or code of bean.java
    Thank you very much!

  21. misfit scrive:
    26 aprile 2013 alle 12:45

    a sample running app would be great. I wonder how to implement the getBufferedData(start, offset) since the interface does not contain a datatype where we can try to extract the start using some kind of iterator perhaps.

  22. nahiko scrive:
    29 aprile 2013 alle 15:42

    Bravo!!! Grande Stefano!!

    It works like a charm!!

    Thank you SOOOO much!!

Leave a Reply

Insert code beetween <code lang="ruby"> and </code>

Copyright 2012 DevInterface s.n.c.

DevInterface Blog is proudly powered by WordPress