Tag Archives: Plugins

How-To: Overcome Rollup Field Limitations with Rolling Batch Processing.. Even In the Cloud

Published by:

Rollup fields are great. But their filters are limited. The most common use case I can imagine is something like year to date sales. But you can’t do that with rollup fields because the filters don’t have relative dates. Want quarter-to-date sales? Sorry folks.

Here’s how to make it happen. This works.. even in the cloud. It should also work in 2011 although I have not tested it there.

In this scenario, we are using a Connection between a Product and an Account to hold quarter to dates sales numbers. I want to regularly calculate the sum of Invoices against the Account for the given Product and save it on the Connection. I’m calling the Connection my “target entity.”

Overcoming the Depth Property and Timeouts

Normally, you could create a recursive workflow that processes something, then calls itself. But you run into a problem with the Depth property; CRM will stop the process after so many iterations and consider it a runaway. So if your workflow calls itself, the 2nd time it runs the Depth property will be 2. After so many times, if Depth = x (I think 15 by default), CRM will kill the process.

The secret comes from Gonzalo Ruiz: the Depth property is cleared after 60 minutes.

The other issue is CRM’s timeouts; you need to make sure the update process doesn’t croak because it’s chugging through too many records.

So we’re going to chunk up our data into 1000 batches and run each batch asynchronously every 61 minutes. A lot of processes doing a little bit of work each. I don’t recommend this with synchronous processing.

The Approach

Here’s the process we’re going to create.

  1. Upon create, assign your target entity a random batch number. I’m using 1000 batches.
  2. An instance of a custom entity contains a batch number (1000 batch controller records for 1000 batches). A workflow fires on this custom entity record every 61 minutes.
  3. The workflow contains a custom workflow activity that updates all target records in its batch with a random number in a “trigger” field.
  4. A plugin against your target entity will listen for the trigger and fire the recalc.

2015-10-06 15_27_28-Drawing1 - Visio Professional

First, create a custom entity. I’ve called mine rh_rollingcalculationtrigger. All you need on it is one integer field.

Now, on your Connection (or whatever entity you want to store the rolling calculated fields), create two fields: rh_systemfieldrecalculationtrigger, and rh_systemfieldrecalculationbatch.

Now create a simple plugin to set the batch number to between 0-999 when the record is created. If you have existing records, you can export to Excel and reimport them with a random batch assignment – the Excel formula randbetween() is great for this.

protected void ExecutePostConnectionCreate(LocalPluginContext localContext)
{
	if (localContext == null)
	{
	throw new ArgumentNullException(localContext);
	}
	Random rand = new Random();
	IPluginExecutionContext context = localContext.PluginExecutionContext;
	Entity postImageEntity = (context.PostEntityImages != null && context.PostEntityImages.Contains(this.postImageAlias)) ? context.PostEntityImages[this.postImageAlias] : null;
	ITracingService trace = localContext.TracingService;
	IOrganizationService service = localContext.OrganizationService;

	// super-simple update
	Entity newConnection = new Entity("connection");
	// Add a random number between 0 and 1000.
	newConnection["rh_systemfieldcalculationbatch"]= rand.Next(0, 1000);
	newConnection.Id = postImageEntity.Id;
	service.Update(newConnection);
}

(Side note: In C#, Random.Next() returns a value exclusive of the upper bound. So each record will get a value 0-999 inclusive.)

Now, we create a custom workflow activity. This inputs the batch number and fires a process called FireBatch. So when this workflow runs on a rh_rollingcalculationtrigger entity with Batch ID = 5, it will call FireBatch against all records with batch ID = 5.

In my case, I like to assemble the service, context and tracing service in the plugin and call my own class.

public sealed class WorkflowActivities : CodeActivity
    {
        /// <summary>
        /// Executes the workflow activity.
        /// </summary>
        /// <param name="executionContext">The execution context.</param>
        protected override void Execute(CodeActivityContext executionContext)
        {
            // Create the tracing service
            ITracingService tracingService = executionContext.GetExtension<ITracingService>();

            if (tracingService == null)
            {
                throw new InvalidPluginExecutionException("Failed to retrieve tracing service.");
            }

            tracingService.Trace("Entered Class1.Execute(), Activity Instance Id: {0}, Workflow Instance Id: {1}",
                executionContext.ActivityInstanceId,
                executionContext.WorkflowInstanceId);

            // Create the context
            IWorkflowContext context = executionContext.GetExtension<IWorkflowContext>();

            if (context == null)
            {
                throw new InvalidPluginExecutionException("Failed to retrieve workflow context.");
            }

            tracingService.Trace("Class1.Execute(), Correlation Id: {0}, Initiating User: {1}",
                context.CorrelationId,
                context.InitiatingUserId);

            IOrganizationServiceFactory serviceFactory = executionContext.GetExtension<IOrganizationServiceFactory>();
            IOrganizationService service = serviceFactory.CreateOrganizationService(context.UserId);

            try
            {
                IWorkflowContext wfContext = executionContext.GetExtension<IWorkflowContext>();
                IOrganizationServiceFactory wfServiceFactory = executionContext.GetExtension<IOrganizationServiceFactory>();
                IOrganizationService wfService = wfServiceFactory.CreateOrganizationService(context.InitiatingUserId);
                int batchId = (int)this.BatchNumber.Get(executionContext);
                Guid thisGuid = ((EntityReference)this.ThisEntity.Get(executionContext)).Id;
                RollupCalculations.FireBatch(service, tracingService, batchId, thisGuid, context.Depth);
            }
            catch (FaultException<OrganizationServiceFault> e)
            {
                tracingService.Trace("Exception: {0}", e.ToString());

                // Handle the exception.
                throw;
            }

            tracingService.Trace("Exiting Class1.Execute(), Correlation Id: {0}", context.CorrelationId);
        }
        [Input("Batch Number")]
        public InArgument<int> BatchNumber { get; set; }
        [Input("This Config Entity")]
        [ReferenceTarget("rh_rollingcalculationtrigger")]
        public InArgument<EntityReference> ThisEntity { get; set; }
    }

FireBatch: query all records with batchID = x and update them with a random number.

// In this case, the processingSequenceNumber is going to be the value from the CRM batch controller entity.
public static void FireBatch(IOrganizationService service, ITracingService trace, int processingSequenceNumber)
        {
            // First, get all Connections with that batch ID.
            String fetch = @"
            <fetch version='1.0' output-format='xml-platform' mapping='logical' distinct='false'>
              <entity name='connection'>
                <attribute name='connectionid' />
                <order attribute='description' descending='false' />
                <filter type='and'>
                  <condition attribute='rh_systemfieldcalculationbatch' operator='eq' uiname='' uitype='product' value='" + processingSequenceNumber + @"' />
                </filter>
              </entity>
            </fetch>";

            // Now do a bulk update of them. 
            EntityCollection result = service.RetrieveMultiple(new FetchExpression(fetch));
            trace.Trace("Processing " + result.Entities.Count + " records on batch " + processingSequenceNumber);

            ExecuteMultipleRequest multipleRequest = new ExecuteMultipleRequest()
            {
                Settings = new ExecuteMultipleSettings()
                {
                    ContinueOnError = false,
                    ReturnResponses = false
                },
                Requests = new OrganizationRequestCollection()
            };

            Random rand = new Random();
            
            int testLimit = 0;
            if (result != null && result.Entities.Count > 0)
            {
                // In this section _entity is the returned one
                foreach (Entity _entity in result.Entities)
                {
                    Guid thisGUID = ((Guid)_entity.Attributes["connectionid"]);
                    var newConnection = new Entity("connection");
                    newConnection.Id = thisGUID;
                    // Note here that we're just dropping a random number in the field. We don't care what the number is, since all it's doing is triggering the subsequent plugin.
                    newConnection["rh_systemfieldrecalculationtrigger"] = rand.Next(-2147483647, 2147483647);

                    UpdateRequest updateRequest = new UpdateRequest { Target = newConnection };
                    multipleRequest.Requests.Add(updateRequest);
                    //trace.Trace("Completed record #" + testLimit);
                    testLimit++;
                }
                service.Execute(multipleRequest);

            }
        }

Warning: use ExecuteMultiple to avoid timeouts.

Now, create a plugin against the Connection to do whatever it is you want. It should fire on change of the rh_systemfieldrecalculationtrigger field.

        protected void ExecutePostConnectionUpdate(LocalPluginContext localContext)
        {
            if (localContext == null)
            {
                throw new ArgumentNullException("localContext");
            }

            IPluginExecutionContext context = localContext.PluginExecutionContext;
            Entity postImageEntity = (context.PostEntityImages != null && context.PostEntityImages.Contains(this.postImageAlias)) ? context.PostEntityImages[this.postImageAlias] : null;
            ITracingService trace = localContext.TracingService;
            IOrganizationService service = localContext.OrganizationService;

            // In this method I do the logic I want to do against the specific record.
            RollupCalculations.SalesToDate(service, trace, postImageEntity);
        }

The final piece is to, well, get it rolling. First, create 1000 instances of rh_rollingcalculationtrigger, setting Batch ID’s 0-999.

Remember, we can create a recursive workflow with the 60 minute workaround. I’m setting it to 61 just to be safe.

2015-10-06 15_09_06-Document1 - Word

Manually fire the workflow once on each of your 1000 recalculation entities. Congratulations, you have perpetual recalculation.

I recommend setting up bulk deletion jobs to remove the system job records this creates. It can be a lot.

Perform a Mileage Expense Calculation Using EasyPlugins

Published by:

We’ve recently deployed a few custom plugins based on the EasyPlugins tool for Microsoft Dynamics CRM 2013 and Microsoft Dynamics CRM 2015, available for free on CodePlex. In this blog we’d like to share some of the things we’ve learned, since the tool is great but not very well documented.

In this blog we’ll review how to calculate an expense mileage reimbursement amount. There are other ways to do this, but this way allows for business users to manage reimbursement rates and effective dates. I’m also making it it slightly complex because I want to demonstrate some of the things that aren’t covered in the official documentation.

In my example, I have:

  • An entity called ics_expense that contains:
    • A date (ics_date) that, by business requirement, the user must be able to leave blank. If they do, the date will default to today.
    • The per-mile rate that was applied (ics_permilerate), a Currency value.
    • A distance value (ics_distance) as a Decimal value.
    • An Amount value (ics_amount), also a Currency, that represents the amount to be paid to the employee.
    • An Option Set (picklist) (ics_type)
      • 100000000 : Itemized Expense
      • 100000001 : Mileage
  • An entity called ics_mileagereimbursementrate, which contains:
    • An effective start date (ics_effectivestartdate) as a Date
    • An effective end date (ics_effectiveenddate) as a Date
    • A reimbursement rate (ics_reimbursementrate) as Currency.

When the Expense is saved, this plugin should:

  • Identify if the user filled in the date (ics_date), if not default it to today.
  • Pull the mileage reimbursement rate that should be applied based on that date. (Design assumption is that there’s only one rate applicable for a given date.)
  • Calculate the amount to reimburse.
  • Fill in the total amount on the Expense but only if it’s of type Mileage.

Here’s the approach:

  1. Stage the data (left side of the EasyPlugins screen)
    1. Pull down the user-input date, the type, and the distance and store them in variables local to EasyPlugins.
    2. Determine if I need to use the user’s date or default it to today.
    3. Pull down the appropriate rate.
    4.  Calculate the total reimbursement.
  2. Update the Expense (right side of the EasyPlugins screen)
    1. Set the Per Mile and total expense fields on the Expense.
    2. Set the date on the Expense if the user left it blank.

Start by creating a new Plugin using EasyPlugins (easyplugins.codeplex.com) – install the package then access it under Settings -> EasyPlugins then selecting New -> Plugin. Give it a name and set it to run PostCreate, Post Update, and Synchronously.

First I need to stage up some data on the left side. First, I’m going to pull down the metadata from the entity. Add Attributes:

  • ics_date – becomes #p1
  • ics_distance – becomes #p2
  • ics_type – becomes #p3

For these, we want the State to be “after action” since we want the value the user actually input.

Now, we determine whether to use the user’s date or ours. To do this, we check if the user input one and if so use that, otherwise make a new one. To do this we use C# syntax in a Calculated field:

#p1 == null ?
DateTime.Today :
#p1

ep-1
Hit Test to make sure your expression is syntactically correct. This will place your new value in #p4.
You may find this syntax unfamiliar. In a more familiar format, it might look like:

if (p1 == null)
#p4 = new DateTime();
else
#p4 = #p1;

Now, find the Expense reimbursement rate to use. We need to find the rate that applied as of the date of the Expense, which is now held in #p4. To do this, we need to find the rate with an effective start date in the past, an effective end date in the future. Construct an Advanced Find like this – the dates you use don’t matter at this point. Make sure the reimbursement rate is in the returned values – add it using Add Columns.

Download the FetchXML. A few edits you need to make:

  1. Delete all attributes except ics_mileagereimbursementrate. That’s the number we’re really after.
  2. In the date fields, replace the value of the date with your calculated date variable, #p4.

It’ll look like this when ready:

<fetch version="1.0" output-format="xml-platform" mapping="logical" distinct="false">
  <entity name="ics_mileagereimbursementrate">
    <attribute name="ics_reimbursementrate" />
    <order attribute="ics_reimbursementrate" descending="false" />
    <filter type="and">
      <condition attribute="ics_effectivestartdate" operator="on-or-after" value="#p4" />
      <condition attribute="ics_effectiveenddate" operator="on-or-before" value="#p4" />
    </filter>
  </entity>
</fetch>

Remember, our assumption going in was that there’s only one reimbursement rate that applies for a given date. So this should only return one record.

Now drop it in a Request parameter. In this case the Aggregate Function is “First” since we are expecting only one record to come back and we want the value from the first returned record.
ep-3
The results of this will be placed in an array called #p5.

The last thing to do is to use this rate to calculate the actual amount to reimburse. Enter a Calculated parameter and use this syntax:

Convert.ToDecimal(#p5[0]) * #p2

We need to be careful here.

  • #p2 is the mileage and is a Decimal value. So to make sure we’re calculating apples to apples, we need to convert the value returned from the query by using Convert.ToDecimal (C# syntax).
  • #p5 is the results of our query. Since we didn’t use an aggregate function, it comes back as an array. Rather than using the variable #p5 straight-up, we need to force it to read the first record, thus the [0] syntax.

ep-4
And that’s #p6, the actual amount to pay the employee.

Phew. A lot of work but our data is now staged and ready to go. To recap:

  • #p1 is the date the user set on the entity.
  • #p2 is the distance traveled as provided by the user.
  • #p3 is the option set that indicates the type of Expense (mileage or otherwise).
  • #p4 is the date the expense occurred – either provided by the user or defaulted to today.
  • #p5 is the array of expense rates that encompass your effective date.
  • #p6 is the total reimbursable amount (distance * rate).

First, we need to update the Expense with the mileage rate and the calculated value. On the right side, Add an Update record.

  • Entity is ics_expense.
  • Where ID = #id (Current ID) – we’re updating the current record, not anything related.

But how do we filter it so that this only happens if the type is Mileage on the Expense? We need to get it to read the Option Set (picklist) value. That’s stored in #p3. Option Sets are managed by EasyPlugins with a Code and a Label attribute. To get at them, you need to use:

#p3.Code

But since we are looking for a Condition here, you need to force the Code value to a String and then check if it’s the value you’re after.

#p3.Code.ToString().Equals("100000001")

Tricky, but that’s what worked for me. Then, just set the value of the amount and the rate.
ep-5
Then, set:

  • ics_amount = #p6
  • ics_permilerate = #p5[0]

What’d we miss? We didn’t set the Date. We need to use a separate Action step because we only want to override it if the user didn’t fill it out, regardless of the type of Expense. (The last Action only applied to Mileage records.)

Last step: make another Update action.

  • Entity = ics_expense
  • Filter = #p1 == null (you can check if a variable is null with a simple == operator like this)
  • Where ID = #id

And set the date field to #p4.
There you have it, folks.

E-Mailing Line Item Details

Published by:

It’s great how everything in CRM uses the XRM object model – it’s so easy to link entities.

But sometimes it can be annoying. Like, what if I want to e-mail a Quote to a customer, or to an internal customer service person who may not be a CRM user? I can’t just drop the line items into an workflow email since it’s a 1:N.

I created a custom workflow activity that does just this. It takes a sales entity’s line items (Opportunity Product, Quote Detail, Order Detail, or Invoice Detail) and assembles it into a string that you can drop into an email. It’s configurable on how it handles bundles and with currency symbols.

Check it out on Codeplex!

Codeplex Project