RSS

Category Archives: IBM

org.openntf.junit.xsp – now with EasyMock support

Frank van der Linden asked in his blog post (http://elstarit.nl/?p=157) if EasyMock or PowerMock can be integrated into the org.openntf.junit.xsp and Ryan J. Baxter mentioned also that a testing framework without a good mocking solution is only the half worth.

Only the half worth is a good reason to complete the effort. So I implemented also EasyMock and PowerMock to the org.openntf.junit.xsp. With version 1.1.0 you get the capabilty to mock objects. You may ask how does EasyMock work. Her a small example:

Imagine this. You have this class called Share. The class can be initialized with a builder method called initFromDocument( Document doc) and you want to test this method.

package org.openntf.junit.example.bo;

import lotus.domino.Document;

public class Share {

    private String m_ShareName;
    private int m_Count;
    private int m_PricePerShare;
    
    public static Share initFromDocument( Document doc) {
        Share share = new Share();
        try {
            share.m_ShareName = doc.getItemValueString("ShareName");
            share.m_Count = doc.getItemValueInteger("Count");
            share.m_PricePerShare = doc.getItemValueInteger("PricePerShare");
        } catch (Exception ex) {
            ex.printStackTrace();
            return null;
        }
        return share;
    }
    
    public String getShareName() {
        return m_ShareName;
    }
    public int getCount() {
        return m_Count;
    }
    public int getPricePerShare() {
        return m_PricePerShare;
    }

    public Object getValue() {
        return m_Count * m_PricePerShare;
    }
    
    
}

Your intention is now to write a test for that (or better you write first the test). But you wont initialize a Notes Document, because one thing that unit testing is about has to do with isolation. At this point comes EasyMock to the game:

package org.openntf.junit.example;

import lotus.domino.Document;

import org.junit.Test;
import org.openntf.junit.example.bo.Share;
import org.openntf.junit.xsp.easymock.EasyMockWrapper;

import static org.easymock.EasyMock.*;
import static org.junit.Assert.*;

public class ShareTest {

	@Test
	public void testShareWithDocumentMock() {
		Document docMock = EasyMockWrapper.createNiceMock(Document.class);
		try {
			expect(docMock.getItemValueString("ShareName")).andReturn("WebGate");
			expect(docMock.getItemValueInteger("Count")).andReturn(5);
			expect(docMock.getItemValueInteger("PricePerShare")).andReturn(2870);
			replay(docMock);
			Share shareWebGate = Share.initFromDocument(docMock);
			assertEquals("WebGate", shareWebGate.getShareName());
			assertEquals(5 * 2870, shareWebGate.getValue());
		} catch (Exception e) {
			e.printStackTrace();
			assertFalse(true);
		}
	}
}

With

Document docMock = EasyMockWrapper.createNiceMock(Document.class);

initialize EasyMock a new document object. You may recognize that instead of calling EasyMock.createNiceMock(), I’m calling EasyMockWrapper.createNiceMock(). This is needed because of security issues.

Now we can “record” what will happen to the docMock object. We have 1 call of getItemValueString(“ShareName”) and 2 calls to getItemValueInteger(). For each call can we define what value should be returned.

expect(docMock.getItemValueString("ShareName")).andReturn("WebGate");
expect(docMock.getItemValueInteger("Count")).andReturn(5);
expect(docMock.getItemValueInteger("PricePerShare")).andReturn(2870);

Like on the tape recorder. It’s time to go back to start. You do this with

 
 replay(docMock);

Now our mocked document object is ready and you can test the initFromDocument method.

There is a lot more to cove about mocking and this only the start.

 

Have fun (and don’t forget to use the EasyMockWrapper)

Christian

 

 
1 Comment

Posted by on September 2, 2014 in Domino, Java, OpenNTF, XPages

 

Performance, some findings!

It’s a long time since my last post. But after my sons heart surgery, my time was consumed by one of the biggest challenge that we as WebGate ever has to face. I will definitely write about this, after the successful closing of the project.

But in the mean time, I have done some experiments with the XPagesToolkit and large datasets. The question that I’ve to answer is very easy. How to process a large dataset like 100’000 documents to a nice List of contact objects, or in java spoken: List contacts = ContactSorageServer.getInstance().getAll().

My first question was of course: Is it really mandatory to have 100’000 Contacts in one List? To be honest, it is mostly no mandatory. Having 100’000 Contacts in a List is a typical “Notes View” case. Most agile Application will not present you a tabular list with 100 results per page and a pager control to browse thru 1000 pages, that you can find the page with the correct dataset. But having such a large dataset helps to find the best solution to read a dataset.

So our test scenario is very easy:

  1. 100’000 Documents to transfer to Contact Object add to an ArrayList.
  2. The transfer from document to object is fix, no changes will make there, we try only to manipulate the way we get the 100’000 Documents and how we can add them to the ArrayList
  3. All tests are instrumented and measured with the XPages Toolbox (see Mastering XPages Second Edition, Chapter 20)

1. Using the current XPT Approach:

The XPagesToolkit uses currently a view and browse the document by getNextDocument(). Each object is added to the ArrayList(). This has the effect that the ArrayList has to extend and realign the internal array each time when we achieve the internal capacity. This is the base value.

2. Using LinkedList instead of ArrayList

The only change to the XPT Approach is, that we use a LinkedList instead of the ArrayList and copy the LinkedList as last opration to an ArrayList. We expect to see a significant speed boost, because the LinkedList can growth in a “one Operation” step and has no reassigning of an internal array.

3. Using a NoteCollection based on the view’s selection formula (suggested by Nathan T. Freeman)

Instead of looping thru a view, we use the views selection formula to build a NoteCollection and access the document directly by “getDocumentById()”. I was personally a bit sceptic about building a selection of NoteID ad hoc. Would this work in such a big dataset?

Each approach was tested 10 times (using the code at the end of the entry). Here the results:

XPT LinkedList NoteCollection
Averag Time (sec) 40.5 39.9 32.7
Min 39 39.2 31.7
Mas 43.8 40.8 33.8
Convesion doc to object (sec) 28.8 29 25.8
getNextDocument() (100’000 calls) (sec) 10.3 10.3
Copy LinkeList to ArrayList (ms) 54
Building NoteCollection (ms) 726
getDocumentByID (100’000 calls) (ms) 2294
getNextNotesID (100’000 calls) (ms) 580

My first finding was, that building a LinkedList and then converting to an ArrayList could be a bit faster than the permanent resizing of the ArrayList, but no so much as expected. That was also the reason to see what was happening in the backend. And as you see here, the time to loop thru the we is long, 10.3 sec, means 0.1ms per document.

But the big surprise was the result with the NoteCollection. It needs only 726ms to build the Collection and other 2.9 sec to browse thru the collection. That’s very fast. This means, even accessing the View to get the selection formula and then building the NoteCollection and browse thru this collection is much faster!

I think that will definitely go in to the XPagesToolkit. But this will also need testing. If you wanna be informed about new SNAPSHOT build of the XPagesTookit, please register here http://mm.wgcdev.ch/mailman/listinfo/xpagestoolkit!

And then, the child in me told me. How will this scale, if you set in the view selection formula a @Today function. We all know that such views has to be rebuilt permanently. And the child wants to play, so I did. Here the same stuff with a view with the following selection formula: SELECT ((Form = “Contact”)) & @Created > @Adjust(@Today;0;0;-10;0;0;0)

The XPT had an average time of 50 sec (that’s what I excepted to see), to get all the document (also 100’000). BUT……..

….. the NoteCollection approach has done this in 32.8 sec! Means near the same speed! The building of the NoteCollection has a time vor 844 ms. That’s definitely great!

Let see what this has for en impact for, small datasets. And a lot of stuff to discuss with the XPagesToolkit Team 🙂

 

Here my test code:

/*
 * © Copyright WebGate Consulting AG, 2014
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at:
 *
 * http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
 * implied. See the License for the specific language governing
 * permissions and limitations under the License.
 */
 package org.openntf.xpt.demo.bo;
import java.util.ArrayList;
 import java.util.LinkedList;
 import java.util.List;
import lotus.domino.Document;
 import lotus.domino.NotesException;
 import lotus.domino.View;
 import lotus.domino.NoteCollection;
import org.openntf.xpt.core.dss.AbstractStorageService;
 import org.openntf.xpt.core.dss.DominoStorageService;
import com.ibm.commons.util.StringUtil;
 import com.ibm.commons.util.profiler.Profiler;
 import com.ibm.commons.util.profiler.ProfilerAggregator;
 import com.ibm.commons.util.profiler.ProfilerType;
 import com.ibm.xsp.extlib.util.ExtLibUtil;
public class ContactStorageService extends AbstractStorageService {
private static ContactStorageService m_Service;
private static final ProfilerType pt = new ProfilerType("ListPerformance");
private ContactStorageService() {
}
public static ContactStorageService getInstance() {
 if (m_Service == null) {
 m_Service = new ContactStorageService();
 }
 return m_Service;
 }
@Override
 protected Contact createObject() {
 return new Contact();
 }
// Performance TestCode
 public List getAllXPT(String viewName) {
 List lstRC;
 if (Profiler.isEnabled()) {
 ProfilerAggregator pa = Profiler.startProfileBlock(pt, "getAllXPT -> " + viewName);
 long startTime = Profiler.getCurrentTime();
 try {
 lstRC = getAll(viewName);
 } finally {
 Profiler.endProfileBlock(pa, startTime);
 }
 } else {
 lstRC = getAll(viewName);
 }
 return lstRC;
 }
public List getAllLinkedList() {
 if (Profiler.isEnabled()) {
 ProfilerAggregator pa = Profiler.startProfileBlock(pt, "getAllLinkedList");
 long startTime = Profiler.getCurrentTime();
 try {
 return _getAllLinkedList();
 } finally {
 Profiler.endProfileBlock(pa, startTime);
 }
 } else {
 return _getAllLinkedList();
 }
 }
public List<Contact> _getAllLinkedList() {
 List lstRC = new LinkedList();
 try {
 View viw = ExtLibUtil.getCurrentDatabase().getView("AllContacts");
 viw.setAutoUpdate(false);
 Document docNext = viw.getFirstDocument();
 while (docNext != null) {
 Document docProcess = docNext;
 docNext = viw.getNextDocument(docNext);
 Contact con = createObject();
 if (DominoStorageService.getInstance().getObjectWithDocument(con, docProcess)) {
 lstRC.add(con);
 }
 docProcess.recycle();
 }
 viw.recycle();
 } catch (Exception e) {
 e.printStackTrace();
 }
 if (Profiler.isEnabled()) {
 ProfilerAggregator pa = Profiler.startProfileBlock(pt, "copyLinkedList");
 long startTime = Profiler.getCurrentTime();
 try {
 return _copyList(lstRC);
 } finally {
 Profiler.endProfileBlock(pa, startTime);
 }
 } else {
 return _copyList(lstRC);
 }
 }
private List _copyList(List linkedList) {
 return new ArrayList(linkedList);
 }
public List getAllNC(String viewName) {
 if (Profiler.isEnabled()) {
 ProfilerAggregator pa = Profiler.startProfileBlock(pt, "getAllNC " + viewName);
 long startTime = Profiler.getCurrentTime();
 try {
 return _getAllNC(viewName);
 } finally {
 Profiler.endProfileBlock(pa, startTime);
 }
 } else {
 return _getAllNC(viewName);
 }
}
public List getAllNCLinkedList() {
 if (Profiler.isEnabled()) {
 ProfilerAggregator pa = Profiler.startProfileBlock(pt, "getAllNC-LinkedList");
 long startTime = Profiler.getCurrentTime();
 try {
 return _getAllNCLL();
 } finally {
 Profiler.endProfileBlock(pa, startTime);
 }
 } else {
 return _getAllNCLL();
 }
}
private List _getAllNC(String viewName) {
 List lstRC = new ArrayList();
 try {
 View viw = ExtLibUtil.getCurrentDatabase().getView(viewName);
 String formula = viw.getSelectionFormula();
 NoteCollection nc;
 if (Profiler.isEnabled()) {
 ProfilerAggregator pa = Profiler.startProfileBlock(pt, "buildNoteCollection " + viewName);
 long startTime = Profiler.getCurrentTime();
 try {
 nc = _buildNoteCollection(formula);
 } finally {
 Profiler.endProfileBlock(pa, startTime);
 }
 } else {
 nc = _buildNoteCollection(formula);
 }
lstRC = new ArrayList(nc.getCount());
 String notesIDNext = nc.getFirstNoteID();
 while (!StringUtil.isEmpty(notesIDNext)) {
 Document processDoc = ExtLibUtil.getCurrentDatabase().getDocumentByID(notesIDNext);
 notesIDNext = nc.getNextNoteID(notesIDNext);
 if (processDoc.isValid()) {
 Contact con = createObject();
 if (DominoStorageService.getInstance().getObjectWithDocument(con, processDoc)) {
 lstRC.add(con);
 }
 processDoc.recycle();
 }
 }
 nc.recycle();
 viw.recycle();
 } catch (Exception e) {
 e.printStackTrace();
 }
 return lstRC;
 }
private List<Contact> _getAllNCLL() {
 List lstRC = new LinkedList();
 try {
 View viw = ExtLibUtil.getCurrentDatabase().getView("AllContacts");
 String formula = viw.getSelectionFormula();
 NoteCollection nc;
 if (Profiler.isEnabled()) {
 ProfilerAggregator pa = Profiler.startProfileBlock(pt, "buildNoteCollection");
 long startTime = Profiler.getCurrentTime();
 try {
 nc = _buildNoteCollection(formula);
 } finally {
 Profiler.endProfileBlock(pa, startTime);
 }
 } else {
 nc = _buildNoteCollection(formula);
 }
 String notesIDNext = nc.getFirstNoteID();
 while (!StringUtil.isEmpty(notesIDNext)) {
 Document processDoc = ExtLibUtil.getCurrentDatabase().getDocumentByID(notesIDNext);
 notesIDNext = nc.getNextNoteID(notesIDNext);
 if (processDoc.isValid()) {
 Contact con = createObject();
 if (DominoStorageService.getInstance().getObjectWithDocument(con, processDoc)) {
 lstRC.add(con);
 }
 processDoc.recycle();
 }
 }
 nc.recycle();
 viw.recycle();
 } catch (Exception e) {
 e.printStackTrace();
 }
 if (Profiler.isEnabled()) {
 ProfilerAggregator pa = Profiler.startProfileBlock(pt, "copyLinkedList");
 long startTime = Profiler.getCurrentTime();
 try {
 return _copyList(lstRC);
 } finally {
 Profiler.endProfileBlock(pa, startTime);
 }
 } else {
 return _copyList(lstRC);
 }
 }
private NoteCollection _buildNoteCollection(String formula) throws NotesException {
 NoteCollection nc = ExtLibUtil.getCurrentDatabase().createNoteCollection(false);
 nc.setSelectDocuments(true);
 nc.setSelectionFormula(formula);
 nc.buildCollection();
 return nc;
 }
 }
 
4 Comments

Posted by on July 19, 2014 in Domino, Java, OpenNTF, XPages, XPT

 

Tags: ,

Automated building and testing of XPages Applications

You know that situation? You have done a roll out of a new version of an application and a short while after being in production, there is this call from an angry customer? A bug, fixed in a previous release did re appear in the new release!

How annoying is that. And you ask yourself DID THEY NOT TEST IT? So if you walk to the QA department and ask nicely about “Testing”, what answer would you get? I thinks that’s the moment when you start to think about “Automated Testing”. And automated testing, without an automated build of your application is nice, but only the half way!

To collect all the pieces to make this happen was a part of my mission to go to IBM Ireland and meet the XPages Team in Dublin. And a day after 3 days of hard work with them, it’s time to talk about automated build and testing of XPages application. First the story line and pre requirement:

  • You are working on a XPages Application.
  • The Application is available as on disk project in a SCM (Source Control Managementsysten)
  • You have build some selenium UI Tests

Each night or on every change to your application that is commited to the SCM, you want to have a automated build of the application and the selenium tests should run.

Thanks to the Headless Designer (see this Wiki about), you can create with a command line directly a new NSF. But there is more to do then only executing a command line. You want to be sure that the designer has all required plugins (like OpenNTF Essentials) during the build process. And all of this plugins should also be removed after a build. After the build you want to have the application moved to a test server and the selenium tests should be executed against this application. The whole build process is now covered by a maven plugin. You can see the progress on that on this GitHub project.

The headlessdesigner-maven-plugin covers now the following steps:

  1. Installing defined features to the designer
  2. Enabling defined features in the designer
  3. Build the application from on disk project
  4. Disabling defined features in the designer
  5. Uninstalling of the features
  6. Copy the build NSF to the target directory of the maven project

The following steps are planed for the next 2-3 weeks:

  • Distributing the NSF to the DominoServer
  • Building a archetype plugin to build per command line a new maven project
  • Thinking about distribution of the plugins to a official maven repository

 

 

 
1 Comment

Posted by on May 1, 2014 in IBM, Java, OpenNTF, Uncategorized, XPages

 

Tags: , ,

Automated build with Jenkins / Some progress

This saturday was dedicated to do some tests with automated build and Jenkins / Maven. Jenkins is a brilliant tool to let a server do some automated tasks. Maven is the tool which delivers a configuration language to describe how to build something. The goal of maven is to do the same thing always in the same way. For me as a highly creative person is doing the same thing always in the same way, a creativity killer. So I think Maven will be my friend, because he is doing all the annoying jobs for me.

But Maven needs some clear instructions to do his stuff right. And like Eclipse needs Maven also a clear environment to work. Based on some research on the SBT Project and with the help of my friend Google, I figured out that I need eclipse.tycho to build plugins, features and update sites. I had prior to saturday already done the following steps:

  1. Setup of Jenkins
  2. Setup of Maven
  3. Setup of Git (for Grabbing the Project direct form the GitHub)

I started with the parent pom.xml file. This file contains all instructions and dependencies for all my plugin. The full file can be read here.

The important thing was to define the following repository:

<repositories>
<repository>
<id>notes</id>
<layout>p2</layout>
<url>${notes-platform}</url>
</repository>
</repositories>
 

As I sad. Maven needs a clear environment to build all the plugins and the eclipse.tycho needs a repository, where all the plugins of the domino runtime are located. Unfortunately such a repository is currently not available. So I had to create a p2 update site. This is in fact the same, as you define your eclipse development environment to use a specific target platform. See the following instruction, how I did that.

As you can see in the repository definition, I’ve nut used in the <url> tag a fix url. It’s a variable, which can be sent to Maven in the command line. This has the good impact that I can use my new update site also as a file resource. To build my POI4XPages Project, I had only to checkout all the code from git, switch to the feature/CIandMavenIntegration branch and go to the root directory of the project:

mvn clean install -Dnotes-platform=file:///D:/updatesite/site.p2/
did the job…. almost 🙂

The next problem was, that Maven uses the standard JDK as Java Environment to compile. So I toke the DominoProg/jvm folder and copied it to my build server. Then I changed the Java Home to my new jvm location. In the mean time, I had also done all the pom.xml files for the plugins. And then, the great moment happens:

buildDoneWhats next….

  • I will now make some documentation, how I’ve setup this also on my Jenkins Server
  • I’ve to think, what should I do with the result of the build (assembly)
  • Test the same with Notes Application (Martin Pradny has made some good progress)
  • And learn how to test after the build 🙂 (The next annoying thing….)

 

 
1 Comment

Posted by on April 7, 2014 in IBM, OpenNTF, Software Entwicklung

 

Tags: , ,

My Slides from Engage.ug

 
Leave a comment

Posted by on March 24, 2014 in Domino, OpenNTF, SocialBusiness, XPages

 

Tags:

Video

Message to the Participant of the German Developer Camp

A short message to the german developer camp (in german).

 
Leave a comment

Posted by on March 19, 2014 in OpenNTF, SocialBusiness, XPages

 

First Swiss XPages Developer Crowd Day

Yes, it was a huge success. The Event-Room of WebGate was full of XPages Developers from Switzerland. One of the reason was, that we had the pleasure to meet Martin Donnelly and Eamon Muldon from the IBM XPages Team. The other reason is the DNA of Domino / XPages Developers. It seems that collaboration and sharing of knowledge is a key factor in this community. So we could have an open discussion across the border and boundaries of company and personal interest. During this special moment we were all only XPages Developers with a passion for good Applications.

Is it the right time to establish an event similar to XPages and Beer, which is a success around the world? Yes we think so. I’m working with several people from the Swiss XPages Developer Community to make this happen. The open discussion and the sharing of knowledge and experience was in the past a success factor and will be in the future a success factor.

Stay tuned for a next update.

I’m currently on travel. I was this week in Geneva and will be next week at the engage.ug and the bccon.de. I would love to meet you there.

And I’m still living without my E-Mail Folders! My life is much easier.

See you!

 
1 Comment

Posted by on March 14, 2014 in OpenNTF, SocialBusiness, XPages

 

Tags: , ,

Offense, Defence and Special-Teams – Thats all you need in XPages!

I’m since the movie “Remember the Titans” a fan of football. A game which can only be won when the teamwork is really great. Because of lack opportunities in Switzerland, I enjoy the fact that each time when IBM Connect is, it is also Superbowl time. This time a great defence won over a good offense.

Inspired by an article of Paul Withers http://www.intec.co.uk/back-end-developers-are-from-mars-front-end-developer-are-from-venus/ I try to explain a good XPages Development Team with some Football words. And football fans forgive me, I’m a swiss guy and I will never have any chance to fit the knowledge of any football fan but….

Front-End Developers / UI Developers are the Offense!

They do the brilliant stuff which makes an application shiny and intuitive. They care about the first impression. They are interested in fast response from the backend, and love to do the most of the application logic (not the business logic) in the client. They measure loading time and render time. They are great in visualisation of data and love a clear interface.

Like in football, these guys makes the points!

Back-End Developers are the Defence!

They do the hard work 🙂 because they build the API for the Front-End Developers to process the data. They care about a robust domain-object-model and a stable API. They care about the transactions to the storage and the processing of the processes. They are more interested in brilliant algorithms than any kind of UI representations.

Like in football, they are responsible that you do not lose the ground.

Interfaces, Integration and Connections – The Special-Teams

Most applications are not islands. They are integrated in business processes and depends on other system. Here comes your Special-Teams to the game. You need these guys who have a deeper understanding of the interfaces and can talk with the “guys on the other side”.

Like in football, the Special-Teams can make the difference! The last point you need to win!

In what kind of developing are you good?

PS: Next Time I write maybe about RAD and Fast-Prototyping 😉

 

Tags: , ,

XWork Server Community Edition

In a time where technology doesn’t matter, its important to have technology easy available. Specially for students and young developers. During the OpenNTF BOF Session at IBM Connect we were shocked how old the youngest XPages Developer was. Are we losing the next generation?

The community around XPages / Domino is one of the most vibrant. There are a lots of good resources available on the internet. But we are missing a licence model which allows young developers easy to learn the technology. What we ask is a XWork Server Community Edition, available for free. The usage should be limited to private person with no commercial aspect. No Replication, Mail, Clustering is needed and the limitations of the regular XWork Server should also be in place.

The core purpose of the XWork Server Community Edition is to attract new developers from outside of the yellow bubble. The should learn the technology and they will love it. I belive that this will lead to new solutions based on the XWork Server, which can be delivered via IBM Softlayer to the cloud. IBM you should give this a try, there is nothing to loose.

 

 
6 Comments

Posted by on February 2, 2014 in IBM, Java, OpenNTF, XPages

 

Tags:

IBM Connections – The next AppDev Plattform?

IBM Connections is a great Plattform to bring people centrict data to a single point of storage and processing. Processing this datas with analytics and bring data, people and the work of them into relation is key to have 360 view of your employee, data and processes. But to get this view, you have to attract developers around the globe to develop applications, which use the power of IBM Connections.
But todays developers are not the same, then 5 years before. They have the power and freedom to choose that framework, which matters for them best. Thats one of the reason why we have such a large and brilliant XPages community. They love the things they can do with one of the most advanced java appdev server!
The IBM SocialBusiness Toolkit Team was wise enougth, to know these fact. And they have made the Toolkit available for a lot of different plattforms. I’m very proud that OpenNTF is the Home of the Social Business Toolkit. And the Toolkit is your key to the IBM Connections Plattform.
So join us (OpenNTF) on wednesday 10 am and learn what the Open Source Comminity for Collaboration Solutions can offer you, for your success. We also have a booth on the Solutions Showcase.