Tuesday, March 31, 2009

K2 Development Methodologies

Jumping straight in to any kind of development without some form of a plan or development structure is dangerous. Most people know this already and are probably saying, well of course. For whatever reason, however, most people don't have a plan or structure when developing a K2 BlackPearl workflow.

Everything is rushed. Right from the Project Manager to the Business Analyst through to the developer, it's all about getting the workflow out the door quickly. This has something to do with the way workflow is pitched to the decision makers. Workflows are easy. Easy correlates to quick. Structured development involves careful planning, development and testing.... that doesn't correlate to quick. So somehow, it seems to me in my experiences with workflow, that some changes need to be made people's mind sets.

Workflow should be treated like any other type of development project. Whether we are talking about Windows Application development or Web Based Application development, we treat this type of development no different.

Whatever your methodologies are, they should be consistent. If they are consistent, then they become easier for your Project Manager to plan around.

I'll share with you mine:

  • Keep your workflow simple;

  • Separate your business rules from your workflow rules;

  • Don't automate poor processes!

These are my philosophies. But what does this mean? What if my process is a complicated one? What are the differences between business rules and workflow rules? What's a poor process, and how do you tell if you have one?

These are good questions. So let's break down what I mean by these three(3) simple statements.

Keep your workflow simple:
Workflow, like with any development application, relies on good planning. A good workflow will not only be easy for users to interact with, but also easy for administrators to manage, and manipulate. This needs to be at the forefront of your mind when designing the workflow. Ask yourself the question: what am I trying to achieve with this process? If you can't say it to yourself quickly and easily, then you are going to get in to trouble. Have a look at breaking your process up in to smaller bites. If you workflow as 40 steps in it, then it is too big. It will be too hard to maintain in the long run. See you can simplify this by creating re-usable bits. Can this be split in to 4 workflows of 10 steps?

An example of this might be a Capital Expenditure Request application. A business might have the need to have one where a line manager has a new starter and he requires that this new starter has a new laptop, mobile phone, chair, desk and personal car. He may need to fill out a form. This form may then need to go through the rigmarole of having it approved by his direct manager. Then approved by the CIO. Then sent to Purchasing to order the equipment. Then to Assets Accounts to register these new purchases, and finally to Internal Deliveries.
Now if we were to design this as a Sequential workflow, this would be fine. We'd have no issues here. There'd be two approval steps, two update steps and and a completion step. Pretty simple, easy to maintain and support.
If, in the same company, we also had a New Starter application. A new person was to commence at this same company. The Line Manager would complete a form that stated the new start's details and position. This new start form would then need to be approved by the line manager's direct manager. The approved by the CIO. Then sent to IT to have his IT Account (email, sign in etc) set up.

I hope you see where I am going with this. Now imagine if the Capital Expenditure (CapEx) application was created 2 months prior to the New Starter application becoming a project. If we create this workflow as another Sequential workflow, we would have the CIO and the direct Manager approving, essentially the same request - one for the person, one for the equipment he'd require. Now, I've met a few CIO's in my time, and that took a lot of careful planning, and a lot of rescheduling. The point I'm making is, these are busy people. Yet in our new workflow application, we've now got him doubling up on his work. I can't see him being too happy with this scenario. In fact, I'd be looking for a new place to work if I kept that up.

What can we do to simplify. We can create 3 workflows here. We can separate out the common steps here in to their own workflow. We can call that the Approval's workflow. All approvals can call this common workflow. We combine the New Starter form with the CapEx form so that the two share these details. When it is launched we kick off the Approval workflow. If it is approved we launch both the CapEx workflow and the New Starter workflow.
This ensures that the CIO only gets one task, but both the new starter and the cap ex are launched as required.
We have more workflows now, but they are smaller, which means each one is easier to manipulate. And each one can be launched on their own. This means they will be re-usable.

Separate your business rules from your workflow rules
This one is a tough one to grasp. Basically what I mean here by business rules are your complex statements of code that are nothing to do with workflow. They are to do with how the business operates. The way I determine the difference is by breaking down where the information required to complete the rule comes from, and how is it used?

I like to keep my workflows dumb. Workflows, in my way of thinking, shouldn't keep business specific data within them. They should only hold information required to make them functional. Anything to do with business related information should be stored within a separate layer.
Have I lost you yet? What am I going on about.
Applications should be broken down in to logical layers. You will have your Data Layer (database - SQL Server, Oracle, may be even SharePoint, SAP etc), your Data Access Layer (the way you will access your Data. Can use LLBLGen, nHibernate or even K2 SmartObjects), your Business Access Layer (a dynamic link library that stores all your complex business rules) and then your Workflow Layer followed closely by your user interface layers.

So going back to my point, work out what layer the information that you are needing to capture belongs to. If it is in your workflow (using the DataFields), make sure that it makes sense to be there. K2 BlackPear out of the box allows an administrator to use the "GoTo" functionality in the Workspace. If you have Business related data that is stored in your workflow, if the user "Jumps" over or to an Activity and misses out on running an important business rule that was only captured in the workflow.... trouble. Your workflow should be dumb. It shouldn't matter where they are in the workflow process, if the "GoTo" functionality is used, it shouldn't break your workflow. The workflow rules should only apply to how an Activity should be interacted with at that point in time. It shouldn't matter what happened before it. That type of information should be captured in your Business Access Layer.
If you are confused still, that's fine. It is quite confusing for most people. Most people will place all their Business Rules in the workflow, behind layers of succeding rules, preceding rules and line rules. All these rules are locked in to that specific space. They are not re-usable, and if there is ever a change in the business rule... hate to be the person that has to fix it.

If you had those types of rules in the Business Access Layer, then if the rule changes, it changes in one place, and your workflow doesn't need to be touched. It's all in one nice neat logical place.

Don't automate poor processes
Can't think of anything worse than being forced to automate a poor business process. To put it bluntly and succinctly, if it doesn't work nicely as a manual process, then it MOST DEFINITELY won't work with ANY kind of workflow tool. I don't care how good a product the workflow tool is, and K2 BlackPearl is one of the better ones, you put junk in, you'll get junk out.

That said, how do we determine how successful a business process will be? How do we pick out the best way. There's no easy answer to this. Many different people have different ways of doing this.

For me, it's all about priorities. What can the client live with, and what can't they live without. That last part is where a base my workflows on. What is the critical path for a successful workflow. What steps must be performed to make the project a winner.

I generally map out the most simple workflow first. The straight line workflow. Go from start to finish with Successes predicted. Get that up as quickly as possible and in front of the users and get feedback. Don't spend time making it perfect, and creating every scenario. Chances are, by the time the Client described the scenario to the Business Analyst, who passed the information in the form of a poorly worded document to the Solutions Architect, who converted it in to technical speak for you, the lowly developer to create... it's already wrong.

Draw a straight line from where your workflow begins, to where your workflow ends, in the quickest way from start to finish, and make that your platform to build on top of. And then get it to the users to view as you are developing. Remember, Junk in means Junk Out. So if it sounds like a complex workflow, ask the question, don't be afraid, is this the best way to do this. Force the Solution Architect to ask the question of the Business Analyst. Force the Business Analyst to go back to the Client. Don't assume they got it right.

Business processes change. The more a company moves in to the automated world, the more their processes, the way they work from day to day, changes. This is inevitable. So design your workflows with the view that this will change. Must change. And therefore whatever you create has to change with it.

These are my methodologies. Whilst it isn't important to follow mine, it is important to follow something. Just jumping in and creating workflows quickly and deploying them live is a recipe for disaster. Just because it can be done, doesn't always mean it should be done.

I promised code... in my next blog, I'm going to start throwing some around. Stay tuned.

Sunday, March 29, 2009

K2 Lessons Learned

It's been well over a year since I have updated this blog... mainly due to neglect, but also due to the lack of any valuable information worth posting.

I have been busy in this space for the last year, K2 BlackPearl development is getting popular. It's fast becoming the buzz word. As such, I've been busy with K2 installs and upgrades.



Upgrading from version 0803 of K2 BlackPearl to version 0807/370 was a big step. This required a bit more foresight than I was told or thought.


  • Install the upgrade over the top of 0803 is easy. Follow the wizard steps and restart your machine. That's what I was told. But I know better now.

What you need to do:
If you have already created a Visual Studio 2005 project in version 0803 and are doing the upgrade in the middle of your development project be careful. You can never be too paranoid in my oponion. Place your project in a nice safe place outside your development machine (don't just rely on Source Control here).


Install version 0807 over the top of 0803. Forget about any updates or patches at this point. Get this upgrade right first. Make sure your project(s) all compile and build in version 0807 first. If you have SharePoint installed, make sure your upgrade on the SharePoint server worked successfully. Especially with the "K2 for SharePoint" page in Central Admin.


Visual Studio Project Issues:


What if your project didn't build. Mine didn't. Don't panic, as it's not as bad as it may seem at first. In version 0803, there were 3 hidden projects that were created under the .kprx project file. Some of you probably already know this.


K2 have now moved these projects to a local destination in your Documents and Settings -> Local Settings section. The reason for this is to allow you to compile your projects while they are in Source Control without your hidden projects permissions blocking them. This is due to those hidden projects being referenced locally, when it compiles the dll's in the hidden projects needed to be checked out to properly build.


Now what happens is that when you open a .kprx file from source control and build it, the file will check the local file system for the hidden projects (which have now been rolled up in to one project) and if they do not exist, they will create a fresh copy, if they do, it will replace it.


This is a big change from 0803 to 0807 if you have already created your projects in 0803 and are converting them. Not all your projects will like this change, especially if you are like me and have referenced your own dll's in your .kprx project. In my case, I had issues with the project properly finding the hidden project on its own. What I had to do was open my .kprx project, go to the references section: Manage Project References, remove the reference to the old Extender Projects altogether and rebuild. I then deleted the Extender Project folder that was created under my project in the file system as it is no longer needed (also another reason for keeping a backup on another server rather than relying on Source Control).


Database issues:


The other change was with the SQL Server permissions. The K2 account on SQL Server in most instances is going to be a db owner on the databases that it controls. In 0807, this is no longer enough. Especially when you are using SmartObjects in your workflow.


Strangely enough, you will be able to Insert new records easily enough, but whenever you try a GetList call, you'll receive a permissions issue. The reason is the new level of permissions required. The K2 account will need SQL Server "sysadmin" rights on some of the databases. I am not sure to which databases as I didn't have enough time to fully test this. But I'd imagine on the SmartBox, SmartFunctions and SmartBroker databases as a minimum.


The reason I was given was due to the new functionality given to SmartObjects. The database can now be placed on a separate database server than K2's databases. This permission level is required for the K2 service to access two SQL Servers across the domain.... I'm not a network or infrastructure person, so please forgive the level of detail here. I just know that if you don't have sysadmin rights on some of the database tables, you're in for a shock.


Back on track:


Once this has been corrected and your Visual Studio projects all build, you've ensured that SharePoint's K2 for SharePoint page is running properly, you can then do your 370 patch update safely. I didn't have any issues with this (make sure you run all your upgrades and patches with the K2 Service Account).