CI/CD - Testing

Dataverse Dynamics 365 Load testing for Model-driven app

julien.biedermann — Fri, 19 Jan 2024 15:00:24 +0000

Dataverse Dynamics 365 Load testing for Model-driven app julien.biedermann Fri, 01/19/2024 - 16:00

Body

Dataverse Dynamics 365 Load testing for Model-driven app

Introduction

In today's digital landscape, where user expectations for seamless, high-performing applications are at an all-time high, ensuring the reliability and scalability of software systems is paramount. Performance testing emerges as a critical practice in the software development lifecycle, aimed at evaluating the responsiveness, stability, and scalability of applications under various conditions.

Performance testing encompasses various types, each serving specific objectives in evaluating the performance characteristics of a system. Here are some common performance test types:

Load Testing: Determines how a system behaves under expected load conditions by subjecting it to a specific number of users or transactions, assessing its response times and resource utilization.
Stress Testing: Pushes the system beyond its normal operational capacity to identify its breaking point and understand how it behaves under extreme conditions, helping determine its resilience and stability.
Soak Testing (Endurance Testing): Evaluates the system's performance over an extended period under a consistent load to detect memory leaks, resource depletion, or degradation of performance over time.
Spike Testing: Assesses the system's ability to handle sudden increases or spikes in load, such as traffic surges, to ensure it can scale appropriately and maintain performance without crashing or slowing down.
Scalability Testing: Measures how well the system can scale up or down to accommodate changes in load or user demand, helping determine its capacity to handle growth or fluctuations in traffic.

In this article, we will focus how to conduct a load testing of a Dynamics 365 Model-driven App.

What is Load Testing ?

Load testing is a crucial aspect of performance testing focused on evaluating how a system behaves under specific levels of concurrent user activity or workload. The primary objective of load testing is to assess the system's performance, reliability, and scalability by subjecting it to simulated loads that represent expected usage patterns in real-world scenarios.

During load testing, the system is subjected to various levels of load, typically represented by a specific number of concurrent users, transactions, or requests per unit of time. By gradually increasing the load, testers can observe how the system responds and identify any performance issues, such as slow response times, errors, or system crashes.

When performing a load test, several important factors should be considered to ensure its effectiveness and relevance to the application's performance goals. Here are some key considerations:

Define Clear Objectives: Clearly define the objectives and goals of the load test. Understand what aspects of the application's performance you are testing, such as response times, throughput, scalability, or resource utilization.
Identify Key Scenarios: Identify the critical user scenarios or workflows that are most representative of real-world usage patterns. Focus the load test on these scenarios to simulate realistic user behavior.
Set Realistic Load Levels: Determine the appropriate load levels based on expected traffic volumes, peak usage periods, and scalability requirements. Ensure that the load levels accurately reflect the anticipated production environment.
Simulate Production Environment: Mimic the production environment as closely as possible during the load test. Consider factors such as network latency, hardware configurations, and software dependencies to create a realistic testing environment.
Analyze Results: Analyze the results of the load test thoroughly to identify performance issues, bottlenecks, and areas for improvement. Look for patterns in system behavior and correlate performance metrics to identify root causes of performance degradation.

How to load test a model-driven app ?

Microsoft is providing the following documentation : https://learn.microsoft.com/en-us/dynamics365/guidance/resources/test-scale-dynamics-365-solution

In this article, we're gonna walk through using those test samples from Microsoft, step by step.

Requirements

Microsoft Dynamics 365 Customer Service environment with Customer Service-Hub model-driven application
At least one user account with multi-factor authentication disabled
Sample data in Dataverse
- Min. 10 active accounts
- Min. 10 active products
- Min. 10 active subjects
Azure subscription (optional): It's a requirement if you plan to load test in Azure Load Testing (https://azure.microsoft.com/en-us/products/load-testing).

1. Download samples files

You will find all the samples files here : https://github.com/microsoft/Dynamics-365-FastTrack-Implementation-Assets/tree/master/Customer%20Service/Testing/At%20Scale/Samples

In this walkthrough, we will use the CreateCase sample. To start, you can download the following files :

CaseCreate.csv
CreateCase.docx
CreateCase.jmx

2. Install Apache JMeter

First, you need to install Apache JMeter https://jmeter.apache.org/usermanual/get-started.html#lets_start.

Apache JMeter is an open-source, Java-based software designed for load testing, performance testing, and functional testing of web applications.

JMeter is not a browser, it works at protocol level. As far as web-services and remote services are concerned, JMeter looks like a browser (or rather, multiple browsers); however JMeter does not perform all the actions supported by browsers. In particular, JMeter does not execute the Javascript found in HTML pages. Nor does it render the HTML pages as a browser does (it's possible to view the response as HTML etc., but the timings are not included in any samples, and only one sample in one thread is ever displayed at a time).

Apache JMeter has two modes, GUI mode and CLI mode.

Use GUI mode for test creation, editing, and debugging.
Use CLI mode to run a load test.

3. Populate CSV file

The file CaseCreate.csv contains the test user's information (username and password). Open the file and add your test user's credentials.

Be careful, if the passwords contain special characters like % or #, it can be misinterpreted during test execution and generate errors.

4. Edit load test script

Run Apache JMeter in GUI mode by opening jmeter.bat in the bin folder.

Open the CreateCase.jmx file.

In Apache JMeter, click File – Open
Use the explorer window to navigate to file or repository containing the CreateCase.jmx file.
Select the CreateCase.jmx file and click Open.

Update User Defined Variables

Click on the User Defined Variables element
Update the following values
1. host – your Microsoft Dataverse environment url (e.g., yourorg.crm.dynamics.com)
2. tenantId - your Microsoft 365 tenantid.
3. cshAppId - the GUID of your Customer Service Hub model-driven application

Update the Thread Group Properties

Click on the CreateCase element
Update the following values
1. Number of Threads (users) – total concurrent users to simulate.
2. Ramp up period (seconds) – duration to increase the threads to the total number of threads.
3. Loop Count – number of times to execute a single thread.

Configure the CSV Data Source

Click on the CSV Data Set Config
Update the filename value with your updated csv file

5. Execute in GUI mode

Test can be started in GUI mode using one of the following methods:

In the Run menu, click Start
Click the Start button in the toolbar
Right-click on CreateCase element and click Start

Before running the test in GUI mode, ensure to reduce the number of threads (users). GUI mode should only be used for test creation, editing and debugging.

You can use the View Results Tree to monitor the progress of the test and to debug, checking the request and response of the calls.

6. Execute in CLI mode

GUI mode has limitations that might impact the performance of a load test. Use CLI mode when executing a load test at full scale.

Test can be started in CLI mode following the steps below:

Open CMD and change the directory to the jmeter bin folder
Enter the following command

jmeter -n -t path\CreateCase.jmx -l path\testresults.csv -e -o path\Results

Option	Definition
-n	This specifies JMeter is to run in cli mode.
-t	Name of JMX file that contains the Test Plan
-l	Name of JTL or CSV file to log sample results to
-e	Generate report dashboard after load test
-o	Output folder where to generate the report dashboard after load test. The folder must not exist or be empty.

Sometimes, JMeter throws a heap dump along with an ‘out of memory’ error while executing the load test. In this case, you need to increase the heap size by editing the jmeter.bat file. The size has been updated to 8 GB in the example below:

if not defined HEAP (
    rem See the unix startup file for the rationale of the following parameters,
    rem including some tuning recommendations
    set HEAP=-Xms1g -Xmx8g -XX:MaxMetaspaceSize=256m
)

6.1 Report

After running the load test using CLI mode with the command above, a HTML report is generated in the folder defined.

This report offers valuable insights into all requests conducted during the testing phase, presenting a comprehensive overview of the test execution. Additionally, it includes a detailed table illustrating the APDEX.

APDEX (Application Performance Index) is a metric used to evaluate the user satisfaction of an application based on its response times. APDEX is a standardized method for measuring and reporting the satisfaction level of users with the responsiveness of web applications or services.

APDEX categorizes response times into three zones:

Satisfied: Response times that meet or exceed the user's expectations.
Tolerating: Response times that are acceptable but not ideal.
Frustrated: Response times that are considered unacceptable by users.

The APDEX score is calculated based on the following formula:

In JMeter, APDEX can be calculated using the following steps:

Define the thresholds for response times that determine whether a sample is categorized as satisfied, tolerating, or frustrated. These thresholds are typically defined based on the application's performance goals and user expectations.
Configure JMeter to track the response times of requests during the test execution.
After the test execution, JMeter calculates the APDEX score based on the collected response times and the defined thresholds.
The APDEX score provides a standardized measure of user satisfaction with the application's performance, helping testers and developers assess whether the application meets its performance goals and user expectations.

By using APDEX in JMeter, testers can quantitatively measure and report the user satisfaction level of web applications or services, allowing them to identify performance issues and prioritize improvements to enhance the overall user experience.

7. Execute in Azure Load Testing

First, you need to deploy Azure Load Testing using your Azure subscription.

Now, before diving into creating a load test in Azure Load Testing with a JMeter script, there are a few configurations we need to tweak:

7.1 Update CSV Data Set Config

In Azure Load Testing the JMX file and all related files are uploaded in a single folder. When you reference an external file in your JMeter script, verify that you only use the file name and remove any file path references.

Remove any file path and reference the filename only in the filename field
Set "Ignore first line" to False
Remove the header row (userName,password) from CreateCase.csv

7.2 Update User Defined Variables

Replace User Defined Variables with the following values :

${__BeanShell(System.getenv("host"))}
${__BeanShell(System.getenv("tenantId"))}
${__BeanShell(System.getenv("cshAppId"))}

7.3 Create and run the test in Azure Load Testing

In Microsoft Azure, navigate to and open your Azure Load Testing resource.
From "Get started" dashboard, select "Create" with the option "Upload a JMeter script" or create it from Tests menu.
On the Basic tab, enter a test name and description
On the Test plan tab, upload both CreateCase.jmx (the one adapted for Azure Load Testing) and CreateCase.csv (without header row)
1. Uncheck "Run test after create" if you don't want to run it directly after creation
On the Parameters tab
1. Add the correct values for the variables host, tenantId and cshAppId
On the Load tab
1. Configure the test engine instances to a meet the target load for the test. The number of threads specified in the jmx file represents the number of thread (virtual users) executed by one test engine instance.
2. Network traffic to Power Platform applications will be routed through Public networks.
On the Test criteria tab
1. (Optional) Define the criteria to specify the performance expectations of the system under test. Use the defined criteria to determine the failure conditions for the test when the criteria evaluate to true.
On the Monitoring tab
1. (Optional) Configure application components to monitor server-side metrics during the test run.
Once all tabs completed, you can review and create your test.
Then, if you have unchecked the box at point 4.1, you need to open and run your test manually
Once the test is executed, you can open it to see the load test results

I'd like to note that I conducted the load test within a trial environment. This could account for the observed performance not meeting expected standards.

Conclusion

I hope you enjoyed this tutorial on running a load test sample in a model-driven app. This article provided an overview of load testing using JMeter and Azure Load Testing, and perhaps it will inspire other bloggers to delve into topics such as:

How to build his own test case in JMeter ?
How to analyze the results ?

In conclusion, it's crucial to recognize that JMeter has limitations when it comes to measuring user experience elements occurring at the client side, such as load time or page rendering. JMeter functions as a performance testing tool but does not emulate a real browser. If we aim to test the user interface (UI) using JMeter, we'd need to incorporate tools like the WebDriver Sampler. This component facilitates integration with the Selenium browser automation framework, enabling more comprehensive UI testing capabilities within the performance testing context. Essentially, JMeter, while proficient at server-side performance assessments, requires additional tools like WebDriver Sampler for thorough evaluation of user interface aspects.

Dataverse Dynamics 365 Load testing for Model-driven app

Image

/sites/default/files/2024-02/article1.png

PowerApps Model Driven UI Testing with Playwright

zsoltzombik — Thu, 07 Dec 2023 15:16:37 +0000

PowerApps Model Driven UI Testing with Playwright zsoltzombik Thu, 12/07/2023 - 16:16

Body

PowerApps Model Driven UI Testing with Playwright

Introduction

In the ever-evolving landscape of software development, test automation has emerged as a pivotal trend, revolutionizing the way we build and deploy software. For businesses embracing Agile and DevOps methodologies, automated testing has become a cornerstone practice, enabling faster development and deployment cycles. In this article, we embark on a journey to explore the profound benefits of automated testing and why it holds such paramount importance in the context of software development, particularly within Dynamics 365 Model Driven Apps. We'll also touch upon the pivotal role of Playwright in automating UI testing within this environment. So, let’s discover the major benefits of automated testing and why automation in software testing matters:

Accelerated Development and Delivery: UI Test automation significantly reduces testing time, as automated tests can be executed rapidly and repeatedly. This results in shorter software development cycles, more frequent releases, quicker updates, and faster time-to-market delivery. The integration of automation, particularly with Playwright, expedites the testing process within Dynamics 365 Model Driven Apps, enhancing the development speed.
Enhanced Productivity: Automated tests operate without human intervention, enabling testing at any time, even during non-working hours. This autonomy minimizes the time software developers and quality assurance teams spend on testing, allowing them to focus on critical tasks. With automation, engineers can prioritize the creation of new features and improvements, ultimately boosting productivity.
Precision and Accuracy: Automated testing reduces the risk of human errors during the testing process. Unlike human testers who may make mistakes, automated tests execute precisely according to predefined criteria. This precision contributes to more reliable and error-free releases, minimizing the risk of failure.
Superior App Quality and Performance: Automated testing offers extensive test coverage, ensuring high-quality and high-performance applications. It allows for simultaneous testing of thousands of test cases across multiple platforms and devices, a feat difficult to achieve manually. With automation, complex and lengthy test cases can be created quickly, elevating the overall quality of applications.
Immediate Feedback: Automated testing provides instant feedback to developers, enabling quick response to failures. This immediate feedback accelerates the bug-fixing process and ensures a better user experience, leading to higher customer satisfaction. In contrast, manual testing can slow down the development and update process.
Enabling CI/CD and DevOps: Test automation is pivotal for implementing Continuous Integration (CI) and Continuous Delivery (CD) practices. In the CI/CD pipeline, every code commit needs to be tested swiftly and efficiently, a task perfectly suited for automation. Automation, including Playwright for UI testing, streamlines the transition to Continuous Testing and Delivery.

Incorporating test automation, especially within the context of Dynamics 365 Model Driven Apps, is not just a trend; it's a necessity. The benefits of automated testing continue to expand as tools and testing frameworks evolve. Beyond traditional automated testing, new areas like model-based testing, predictive analysis, robotics automation, and API test-case automation are emerging. To stay at the forefront of technological evolution, implementing test automation is imperative. It accelerates development, enhances efficiency, accuracy, and productivity, all while maintaining the highest standards of app quality and performance. Partnering automation with a robust testing solution and integrating it with your CI/CD tools ensures that you harness its full potential efficiently on real devices, propelling your software development to new heights without compromising on quality or performance.

Setting Up Your Environment for Playwright in Dynamics 365

Prerequisites for Playwright Setup

Before diving into using Playwright for UI testing, ensure you have the following prerequisites in place:

Native Installation

Playwright Test for Visual Studio Code: This is available on Visual Studio Marketplace. Make sure you have version 1.19 or newer to leverage the full capabilities of Playwright in Visual Studio Code.

Playwright NPM Package

Cross Platform Support: Playwright is designed for versatility, supporting Windows, Linux, and macOS. It can be used for testing both locally and on Continuous Integration (CI) platforms such as Azure DevOps, GitHub.
Browser Testing Flexibility: It offers extensive testing capabilities, including both headless and headed modes. For mobile testing, Playwright provides native mobile emulation for Google Chrome on Android and Mobile Safari, ensuring comprehensive coverage across devices.

Azure Subscription (Optional)

Enhanced Testing with Microsoft Azure: While not mandatory, an active Microsoft Azure subscription is recommended for a more robust testing framework. It becomes essential if you're planning to conduct automated tests within Azure or integrate load testing into your CI/CD pipelines. Utilizing Azure enhances your testing scope and efficiency, particularly for complex Dynamics 365 environments.

Having these prerequisites in place sets a solid foundation for effective and efficient use of Playwright in your Dynamics 365 UI testing projects.

Installation and Configuration

Before you start, you will need install the following:

NodeJS - https://bit.ly/getnodejs
VSCode - https://bit.ly/downloadvscode

When installing Node, select the LTS version and accept the defaults.

After installation, you should check you have Node installed correctly, at command line type:

npm

You should see the npm command help information displayed.

To check you had the VSCode installed correctly, at the command line type:

Code .

This should open VSCode at the code folder location you are currently in.

UI Test Project folder setup

Playwright UI Test project do not have a project file like C# (.csproj) so you can simply create a new folder with the name of your project. I will use UITest.

At the command line, type:

Mkdir UITest
Cd UITest

Install Playwright Test for Visual Studio Code

Open Visual Studio Code.

Select Extension, then search for Playwright Test for VSCode.

Then click on Install.

Open the command panel (Ctrl + Shift + P), and then enter the following command: Install Playwright.

On the next screen, select the browser(s) you want to run your tests in. You can change the setting later in the playwright.config file.

Finally you created your first Playwright test project.

Get a Test Environment

Next, you need a Dynamics 365 environment that you can run tests in. If you're finalizing an implementation project, you probably have a test environment already. If you don't, you can get a free 30-day trial (for this article, we provisioned a trial Dynamics 365 Customer Service Environment.

Setting up Test Environment

To optimize the test environment for automated testing, especially when using Playwright with a model driven app, follow these steps:

Disable Security Defaults

Adjust Security Defaults: For the purpose of automated testing it's necessary to turn off security defaults. This ensures that the automated tests can run without unnecessary interruptions interruptions or blocks, which are typically in place for security reasons. Please follow this link: https://learn.microsoft.com/en-us/entra/fundamentals/security-defaults

Create and Configure User Accounts

Access Microsoft 365 Admin Center: Utilize the Microsoft 365 admin center to set up the required user account(s) for testing.
Assign Security Roles: Properly assign the necessary security roles to these accounts to ensure they have the appropriate access levels for testing.

By carefully configuring the test environment as described, you create a more controlled and effective setting for conducting automated tests using Playwright in Dynamics 365.

First Playwright Test

Let's create our first test! Playwright provides convenient utilities for capturing your browser interactions and generating test scripts. To accomplish this, simply execute the Playwright Node command with the codegen argument. When you include the URL of your model-driven Power App as a parameter, it initiates a browser session that records the user's actions within the Playwright Inspector.

To inititate the recording a new test, type this command on Visual Studio Code Terminal window:

npx playwright codegen https://<you-crm-org>.crm4.dynamics.com/main.aspx?appid=<app guid>

Playwright open a new browser instance and launch Playwright Inspector:

Enter the credentials.

Then select Service -> Cases.

Click on New Case button.

Fill the form, then click on the Save button.

Finally, take a look at the Playwright inspector window, it generated the code of our test.

After the test script generated, we need to create a new test file under the tests folder such as and copy the code generated by Playwright Inspector.

Select the Test icon on Visual Studio Code.

Find the test we created previously.

Click in Debug Test button.

It kicks off by running tests in debug mode, and eventually, you'll spot the test results right in the terminal window.

Here's a small preview of the upcoming articles on Playwright.

The upcoming article, Writing Effective UI Tests with Playwright in Dynamics 365 will focus on mastering Playwright's testing capabilities, including automated browser testing and managing dynamic content. It will provide a step-by-step guide to developing test scripts specifically tailored for Dynamics 365 UI, complete with example scripts and scenarios, and conclude with strategies for debugging and troubleshooting these tests.
Third article Advanced Playwright Features for Dynamics 365 Testing will cover sophisticated techniques for working with multiple browsers and devices, including cross-browser and responsive testing. It will also delve into Playwright's advanced capabilities like network interception, mocking, and visual regression testing, along with their integration in CI/CD pipelines.
Fourth article titled Best Practices and Optimizing Test Performance focuses on creating maintainable test code with well-organized structure and reusable components. It will discuss strategies to enhance test performance, such as parallel testing and methods to reduce test flakiness, and will conclude with approaches for monitoring and reporting test results.

PowerApps Model Driven UI Testing with Playwright

Image

/sites/default/files/2023-11/1970%27s%20sci%20robot%20testing%20software%2C%20studio%20light%2C%20on%20an%20emerald%20colored%20background.jpg

Error handling in Power Automate flows

Sergio Ferraresi — Tue, 30 May 2023 08:22:56 +0000

Error handling in Power Automate flows Sergio Ferraresi Tue, 05/30/2023 - 10:22

Body

Error handling in Power Automate flows

Low Code solutions using Power Automate flows are becoming more and more popular in CRM Dynamics projects. But often low code solutions show maintenance issues because of the tendency of the developers (myself included of course) to neglect the exception handling aspects of the power automate flows under the assumption that "low code" also means "low error handling".

Our objectives

With this article I will cover the topic of how to handle errors in Power Automate flows and how to track them for a later analysis of them. What we need to achieve is a way of:

capturing the errors that have been raised
identifying the action that has failed
getting a description of the reason for this failure

Let's get started.

Out-of-the-box features: some limitations

We will first have a look at the out-of-the-box features offered by Power Automate to check the flow executions.

When you need to verify when a flow has run and whether the flow execution was successful or not, you open the flow and you are presented a run history reporting the flow executions of the flow in the last 28-days:

In this view you do not get any details about the reason for the failure. You need to open the specific execution that failed by double clicking on it and you get the reason for the failure in the "Error Details" pane on the right hand side of the run view.

This way of inspecting the errors requires a periodical check of the flow executions: a time consuming action that can be easily avoided with the following solution.

Handling errors occurring in one action

Let's suppose to have a flow responsible for updating the owning team of an account according to the country where the company is based in.

We are going to trigger the flow when a new account is created or when its address is modified.

If the company address contains a USA address, the flow assigns the company to the USA Team, otherwise it will be assign to the Non-USA Team.

The flow structure is the following one:

If the update of the account owning team is successfully performed we want to send an email to confirm the successful assignment of its owner.

We also want to handle possible problems occurring in the update action called "Update Account owning team": in case an issue occurs while updating the account, we do not want the email to be sent, instead we want to catch the error, get its details and log the error for a later analysis.

At the moment we suppose to log the error in the Dataverse: I created a custom table called "Error Log" with the needed fields to store the error details. I will propose a slightly different solution later in the article.

The first thing we need to do to handle the update errors is to create a parallel branch after the update action as shown below.

After adding the branch we will have two actions running in parallel after the update action.

Our objective is to tell the flow to execute the branch on the right (Log error branch) only in case an issue with the previous action, "Update Account owning team", occurs. To achieve this we use the feature "Configure run after" of the "Log error" action.

Using Configure run after you can tell the flow to run the specific action if and only if an error has occurred in the previous action. You can configure “Run After” to almost all Power Automate actions by simply clicking on the three dots on the right corner of the action which you wish to configure and select “Configure run after” from the drop-down menu.

From there, you can select either one of the options or multiple options.

Is successful – If the action runs successfully.
Has failed – An action has any type of failure (except timeout).
Is skipped – An action was skipped. Actions are skipped either when a condition is not met, or when a previous action before that fails.
Has timed out – An action times out. This can happen if the call to the backend times out (120 seconds), or for long-running actions such as approvals, after 30 days.

We want to determine what the flow has to do when our update fails: we then select the "Has failed" option.

The error handling branch is displayed in the flow designer with a dotted line:

At this point our flow will send the email when the assignment is successfully performed and it will execute the "Log error" action if the update fails.

Having handled the possible update errors in the Log error branch, this flow will result to have successfully run even if an error while updating the account occurred.

To avoid this and ensure that the failed flow is marked as "failed" in the run history, add a Terminate action at the end of the error handling branch as shown in the picture below.

What we are still missing now is how to get the error details to be logged into the Dataverse.

To get this information we can rely on built in functions. The function we need in this case is called actions: This function takes the action name as input parameter and it returns an action's output at runtime in JSON format. The JSON format has approximately the following structure (further fields could eventually be present):

{
  "name": "Update_Account_owning_team",
  "inputs": {
    "parameters": {
      "entityName": "accounts"
    }
  },
  "outputs": {
    "statusCode": 400,
    "body": {
      "error": {
        "code": "0x80040203",
        "message": "Attribute: ownerid cannot be set to NULL"
      }
    }
  },
  "startTime": "2023-05-10T12:08:27.6333954Z",
  "endTime": "2023-05-10T12:08:49.3524607Z",
  "code": "BadRequest",
  "status": "Failed"
}

From the JSON structure of the returned value I will extract the error message and I will put it into the "Error message" field of my custom entity "Error log" as shown in the picture below.

As the error message is nested in the JSON message, we need to use the following syntax to extract it :

actions('Update_Account_owning_team')?['outputs']?['body']?['error']?['message']

In our custom entity "Error log" we will now find all the errors occurred while updating the owning team of the accounts and the specific error message.

Handling errors in multiple actions

So far we handled the errors that could happen in a single action. A flow usually consists of a lot of actions and handling the errors using "Configure run after" for each action in the flow is not an efficient approach. In order to be able to handle the errors occurring in multiple actions the best solution is to group the actions using the so called Scope blocks.

In the scope action I can nest all my flow actions (the initialisation of the variables cannot be nested into a Scope action and must be kept outside, before entering the scope) as shown below. I will call this scope block "Try" (according to the error handling patterns in other programming languages).

After the Try block I will then add another Scope action that I will call "Catch": on the Catch action I will then set the "Configure run after" exactly as I did for the single action before. The Catch block will then be executed only if one or more actions included in the Try block fail.

In the Catch block we now need to get the details of what happened in the Try block: as in the Try block several actions could have thrown an error, we cannot use anymore the actions function as it refers to a specific action. Instead we will use the expression function called result. This function returns an array of objects consisting of the Input / Output information of the top-level actions in the specified scoped action. I now want to capture the Output of the actions in the Try block that have failed.

To do it I need to filter the array of objects returned by the "result" function to get the objects of the failed actions only. This can be achieved using the Filter array action. Finally we can use the action "Create HTML Table" to store the returned results in a structured way. We will rename this action "Select result".

The result expression is result('Try'), the status expression: item()?[status] while the Body is body('Filter_array_-_Status_=_Failed').

Once you have all your information stored in the "Select result" action you are ready to use it and put it into the proper fields of the Error Log record.

To include information about the flow in your error log, such as the name of the flow, you also need to use the power automate function called workflow. This function returns the Id, the name, the type and some other information about the running flow. We will use it for getting the flow name and for retrieving the flow information needed for building the Url of the specific execution or run where the log was written (you can find the syntax of it below).

This is how our catch block looks like:

Here the expressions of the fields above:

Action : outputs('Select_Result')?['body']?[0]?['Action']

Error message: outputs('Select_Result')?['body']?[0]?['Message']

Execution Url: concat('https://flow.microsoft.com/manage/environments/', workflow()?['tags']['environmentName'], '/flows/', workflow()?['name'], '/runs/', workflow()?['run']['name'])

Using Flow templates

When you create a new flow you get an empty page without any of the logic we just described. In order not to write the same error handling code every time you create a new flow you can prepare a flow including the Try-Catch pattern we just explained and use this flow as a template when you create a new one.

This is how the template flow would look like:

The Try block will contain the whole flow logic with its different actions.

The Catch error block can be the same for every flow and does not need to be changed when you start to develop a flow: this block will be like the one presented in the previous picture..

I also added a Catch success block where you can put the actions you want to take when the flow has successfully carried out its job, as sending an email or setting some flag or status to completed on other tables. You might want to track the executions of the flows in both cases, either it fails or it successfully runs: in this case you can log it in the same table with different status in the two catch blocks.

I mentioned before that using a custom entity to keep track of the flow failures is not an ideal solution because the Dataverse can quickly increase its size with a lot of error messages. An alternative solution would be using a Power BI Data Set or an external database for storing the error records, depending on your storage availability.

Well, we've come to the end of this article: I hope you enjoyed reading it and that it can be useful for your projects.

Meet you in the next Chronicle!

Error handling in Power Automate flows

Image

/sites/default/files/2023-05/Error%20handling%20in%20Power%20Automate%20flows.jpeg

Power Platform Managed Environments and Pipelines

mario.ortegaaguayo — Thu, 09 Feb 2023 12:39:41 +0000

Power Platform Managed Environments and Pipelines mario.ortegaaguayo Thu, 02/09/2023 - 13:39

Body

Power Platform Managed Environments and Pipelines:

What is Managed Environments?

Managed Environments is a suite of capabilities that allows system administrators to manage Power Platform with more control, less effort, and more insights. Administrators can use Managed Environments with any type of environment except the Developer environments.

Once enabled, users will need a Power Apps, Power Automate or 365 license Premium to use respective resources in this environment.

Managed Environments from an IT perspective allow administrators to get better insights and feedback into how the environment is used.

The principal features that Managed environments offer to administrators are:

Weekly digest: system administrators will receive by email analytics about top apps, most impactful makers, and inactive resources you can safely clean up. Weekly digest will be delivered to administrator mailboxes once a week.

Sharing Limits: Administrators can limit the number of users who can share canvas apps.

Data Policies: Administrators can easily identify all the data policies that are applied to an environment. Data policies define the consumer connectors that data can be shared with. They make sure that data is managed in a uniform manner across your organization.

Solution checker: With the solution checker feature, you can perform a rich static analysis check on your solutions against a set of best practice rules and quickly identify these problematic patterns. After the check completes, administrator receives a detailed report that lists the issues identified, the components and code affected, and links to documentation that describes how to resolve each issue.

Solution checker analyzes three solution components:

Dataverse custom workflow activities.
Dataverse web resources (HTML and JavaScript).
Dataverse configurations, for instance SDK message steps.

How to enable Managed Environments

In the Power platform admin center, in the left panel, select Environments.
Select the check mark to the left of an environment.
On the command bar, select Enable Managed Environments. If the environment is already managed, select Edit Managed Environments.
Configure the settings, and then select Enable or Save.

It needs to be highlighted three points regarding user permissions.

You need to be Administrator (role of Global administrator, Power Platform service administrator or Dynamics 365 administrator in Azure Active Directory.
Any user with permissions to see the environment details can view the Managed Environments properties for an environment.
Users with the Delegated Admin role or the Environment Admin security role are not allowed to change the Managed Environments property in an environment.

What are pipelines in Power Platform?

Pipelines in Power Platform aim to democratize application lifecycle management (ALM) for Power Platform and Dynamics 365 customers by bringing ALM automation, continuous integration and continuous delivery (CI/CD) capabilities into the service in a manner that is more approachable for all makers, admins, and developers.

Pipelines significantly reduce the effort and domain knowledge previously required to realize the ROI from adopting healthy, automated ALM processes within your team or organization.

Pipelines offer the next advantages:

Admins easily configure automated deployment pipelines in minutes rather than days or weeks.
Makers have an intuitive user experience for easily deploying their solutions.
Professional developers can (optionally) run pipelines using their preferred tools such as the Power Platform command line interface (CLI).

How to configure pipelines

Before starting to configure pipelines, we need to clarify a few points:

It is necessary to create a host environment, known as Pipeline Orchestrator, which is a special purpose environment which acts as storage and management layer for all pipeline configuration security and run history. The recommendation is to create this environment as a dedicated production environment separate from the other environments and tenants.
Environments that would be configured to be part of pipeline experience must have the same Region as the “Pipelines host”
Pipeline Orchestrator "CAN BE" Managed Environment but the rest of the environments "MUST BE" Managed Environments.
In Pipeline Orchestrator environment, you need to install "Power Platform Pipelines" application as resource. You can install it in the next path: Power Platform Admin Center-> Environments -> Pipeline Orchestrator -> Resources -> Dynamics 365 apps -> Install App -> Power Platform Pipelines.

Once installed, open "Pipeline Orchestrator" environment and in Pipeline Setup -> Environment you will create a Dev Environment and Prod Environment linked to the original environment through the ID.

Below you have the configuration for each environment. Highlight that you can get "Environment ID" from the admin center, just by clicking on the required environment.

It is also worth mentioning that, in the field "Environment Type", is required to specify which is the purpose of the environment. In this case, "Dev Environment" is the data source, therefore is a "Development Environment", and "Prod environment" will be the "Target Environment" or the environment where the data will be deployed.

The next step is to create the Pipeline. As you see in the picture below, you click on "New" for navigating to the Pipeline configuration.

After that, provide a name and in "Linked Development Environments", you can add an existing development environment, in this case, the one you created earlier.

To conclude the Pipeline setup, you need to create a "New Deployment Stage". Clicking on this button, a pop-up will be opened for the Deployment Stage creation. Add Prod Environment in "Target Deployment Environment ID", then click on "Save and Close". The configuration is now complete.

The final step is to deploy a solution from Dev environment to Prod environment.

For this deployment, you have to navigate to your solution created in Dev Environment. As seen below, a new section is created and you can now select the previously created Pipeline "Testing Pipelines". By clicking on "Deploy here", the solution will be deployed from the Dev environment to the Prod environment.

Select your Pipeline and Stage (if you have only one Stage and Pipelines, it is selected by default) and click "Next".

After the validation, a Summary will be showed. Here you can specify the Version and review some configuration fields.

Click on "Deploy" for finishing the deployment process.

In conclusion, Microsoft has released a very useful tool for Power Platform developers and administrators. We will soon be able to see the final release and what functionalities they add.

Power Platform Managed Environments and Pipelines

Image

/sites/default/files/2023-02/Standard-DevOps-CICD-1024x483.png

Handle secrets in Azure DevOps

Mattias Liloia — Tue, 31 May 2022 12:18:54 +0000

Handle secrets in Azure DevOps Mattias Liloia Tue, 05/31/2022 - 14:18

Body

When creating a ci/cd pipeline for your project, at some point you have to define a connection to your environment. In case of Dataverse, the connection string will contain clientid and client secret values. It's always a good idea to store secret values in a secure place, instead of putting them in clear text into your pipeline definition file (yaml) and potentially pushing them into your code repository.

Azure DevOps provides you a number of possible solutions to address just that:

Azure DevOps Service Connection: https://docs.microsoft.com/en-us/azure/devops/pipelines/library/service-endpoints
Azure Key Vault: https://docs.microsoft.com/en-us/azure/devops/pipelines/release/azure-key-vault
Azure DevOps Secret Variables: https://docs.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#secret-variables

Personally, I prefer to use Azure Key Vault because it allows me to use these secrets in other applications too (like for example an Azure Functions).

Power Apps Portal source code with Visual Studio Code and Power Platform CLI

Fabian Kalchofner — Tue, 18 Jan 2022 09:45:49 +0000

Power Apps Portal source code with Visual Studio Code and Power Platform CLI Fabian Kalchofner Tue, 01/18/2022 - 10:45

Body

Power Apps Portal source code with Visual Studio Code and Power Platform CLI

When developing a Portal with Power Apps Portal, you'll reach a point, where you want to have your developments in a source control or where you want to deploy your portal to a production environment.

With the help of the Power Platform CLI, you can do all this tasks directly from within Visual Studio Code. You even have the possibility to adjust content (e.g. content snippets or web templates) directly in Visual Studio Code and to update them on your development environment.

You'll need the following tools to start:

Visual Studio Code
Power Platform Tools
- This is an extension of Visual Studio Code, which also installs the Power Platform CLI

Connect to Dataverse

Once you installed the tools, you must connect to a Dataverse Environment with the Power Platform CLI. You can either make a connection with app secrets or client credentials:

pac auth create --name "$ConnectionName" --url "$url" --applicationId "$appId" --clientSecret "$clientSecret" --tenant "$tenantId"

pac auth create --name "$ConnectionName" --url "$url" --username "$username" --password "$password"

Open a terminal session in Visual Studio Code and execute the command. Remember the name of your connection, as you will need it later on.

To switch between different environments, you can execute the following command:

pac auth select --name "$ConnectionName"

Download Portal Content

To download existing content from a portal, execute the following command in the terminal session:

pac paportal download --webSiteId $WebsiteId --path "$DestinationFolder" --overwrite true

The $WebsiteId is the GUID of your portal website record. Within the $DestinationFolder a folder with the name of the website will be created.

You can limit the the entities to download by adding the following parameter:

--includeEntities "adx_entityform,adx_entityformmetadata" --excludeEntities "adx_weblink,adx_weblinkset"

After executing the command, your destination folder will look like this:

Upload Portal Content

You can create different deployment profiles to have different values for each environment. Do to so, create a folder (if it does not exist) "deployment-profiles". In this folder, you can create a yml file for each environment, e.g. "production.deployment.yml". Within the yml files, you can define the values:

Each record can be updated like this. Just provide the entity name, the record id and the attributes to update.

To upload the content, execute the following command:

pac paportal upload --path "$SourcePath" --deploymentProfile $DeploymentProfile

$SourcePath contains the relative path to the website folder from your export. The $DeploymentProfile contains the first part of your yml file (e.g. "production").

You can modify content and upload it to the developer environment if needed.

Source Control

As all exported files are either yml, js, or html files, you can simply add the exported portal content to any source control system.

Power Apps Portal source code with Visual Studio Code and Power Platform CLI

Dataverse Plugin telemetry with Azure Application Insights

Danny Rodrigues Alves — Thu, 28 Oct 2021 14:44:48 +0000

Dataverse Plugin telemetry with Azure Application Insights Danny Rodrigue… Thu, 10/28/2021 - 16:44

Body

Dataverse Plugin telemetry with Azure Application Insights:

Introduction

Troubleshooting a piece of code is probably one of the main task in software development. With good logs, you can understand what happened and the code behavior. This is even more accurate for a CRMs and Microsoft Dataverse, where non-IT users are working with the CRM all day long.

In Dataverse and Dynamics 365, tracing has been the way to record run-time information and assist us to diagnose behaviors. Trace logs are written directly to a Dataverse table - PluginTraceLog - and triggered by a trace system settings (off, on or only for exceptions). These logs must be written specifically by the developer in the C# code. An example of such code is given below in the Plug-in Tracing via Trace section.

In July 2021, Microsoft has announced a public preview version to have plug-in trace logs in Application Insights. Application Insights is a feature from the Azure world used to monitor and capture all enterprise activities. The data is collected into Azure Monitor and is used for performance diagnostics, alerts, visualizations, failures reviews and many more. You can created dashboard, enabled Smart Detection or even create you custom queries to asses the system's health. Microsoft describes in more detail the benefits of such tool. Combining this with other tools like the model-driven app monitoring give a full picture of the plateform's health.

With this integration, there are no SDK or other work needed from the developers, everything is already set up and ready to use with the ILogger interface.

Set up insights

To set up the integration, the first thing to do is to create an Application Insights resource in Azure. This is very simple to do:

Log in the Azure portal
Create a new resource
Search for Application Insights and click Create
Fill the fields and create the new resource
Note that you must have at least one of the following rights on this resource: contributor, writer, or admin.

Once the Azure resource has been created, you need to go to the Power Platform admin center:

On the right panel, select Analytics > Data export
Click on the button New data export and select the desired environment
Click Next
Select the Application Insights resource newly created
Click Create

And that's it! Nothing more to do.

If for whatever reason you decided to not use the Application Insights telemetry, simply deleting the Data export will do the trick.

Telemetry with Application Insights

With this new integration, all logs a developer writes within the ILogger interface from the Organization Service SDK are available in the Application Insights resources specified, without the need to code anything more! Under the hood, Dataverse will receive the logs and send it to the Application Insight resource, meaning that some delay might be experienced before the telemetry is visible in Azure.

Let's build a plug-in that makes usage of the ILogger interface. Please check below on the section Plug-in Tracing via Trace on how to register and trigger this plug-in.

using System;
using System.ServiceModel;
using Microsoft.Xrm.Sdk;
using Microsoft.Xrm.Sdk.PluginTelemetry;

namespace DynamicsChroniclesPlugins
{
    public class FollowUpMeetingPlugin : IPlugin
    {
        public void Execute(IServiceProvider serviceProvider)
        {
            // Obtain the ILogger service
            ILogger logger = (ILogger)serviceProvider.GetService(typeof(ILogger));

            // Obtain the execution context from the service provider.  
            IPluginExecutionContext context = (IPluginExecutionContext) serviceProvider.GetService(typeof(IPluginExecutionContext));

            // The InputParameters collection contains all the data passed in the message request.  
            if (context.InputParameters.Contains("Target") && context.InputParameters["Target"] is Entity)
            {
                // Obtain the target entity from the input parameters.  
                Entity entity = (Entity)context.InputParameters["Target"];

                // Obtain the organization service reference which you will need for web service calls.  
                IOrganizationServiceFactory serviceFactory = (IOrganizationServiceFactory)serviceProvider.GetService(typeof(IOrganizationServiceFactory));
                IOrganizationService service = serviceFactory.CreateOrganizationService(context.UserId);

                try
                {
                    using (logger.BeginScope("Init phase"))
                    {
                        logger.LogInformation("FollowupPlugin: Begin the plugin.");

                        if (entity.Id == null || entity.Id.Equals(Guid.Empty))
                        {
                            logger.LogWarning("Plug-in registered for a wrong entity {0}", entity.LogicalName);
                        }
                    }
                    
                    // Create a task activity to follow up with the account customer in 7 days. 
                    using (logger.BeginScope("CreateTask"))
                    {
                        Entity followup = new Entity("task");

                        logger.LogInformation("FollowupPlugin: Building Task record...");
                        followup["subject"] = "Send e-mail to the new customer.";
                        followup["description"] = "Follow up with the customer. Check if there are any new issues that need resolution.";
                        followup["scheduledstart"] = DateTime.Now.AddDays(7);
                        followup["scheduledend"] = DateTime.Now.AddDays(7);
                        followup["category"] = context.PrimaryEntityName;
                        followup["regardingobjectid"] = new EntityReference(entity.LogicalName, entity.Id);


                        // Create the task in Microsoft Dynamics CRM.
                        logger.LogInformation("FollowupPlugin: Creating the task activity.");
                        Guid taskId = service.Create(followup);
                        logger.LogInformation($"FollowupPlugin: Task created with GUID = {taskId}");
                    }
                }

                catch (FaultException<OrganizationServiceFault> ex)
                {
                    throw new InvalidPluginExecutionException("An error occurred in FollowUpPlugin.", ex);
                }

                catch (Exception ex)
                {
                    logger.LogError(ex, "FollowUpPlugin failed");
                    throw;
                }
            }
        }
    }
}

Application Insights Logs

Once the plug-in is registered and triggered, you can go to the Application Insights Logs console. On the portal.azure.com, you can open the Application Insights resource.

Then to see the telemetry, open the button Logs on the right panel, under the Monitoring section.

In the traces table of the Application Insights, all logs can be found. In the above image, you can see the 4 logs entry.

In the trace, all information is stored, from the message to the scope - or customDimensions and OriginalFormat - and the timestamp.

Application Insights Tables

The traces table is not the only one that can be queried. In total, there are 10 tables

availabilityResults
- Summary data from availability tests
browserTimings
- Data about client performance, such as the time taken to process the incoming data
customEvents
- Custom events created by your application
customMetrics
- Custom metrics created by your application
dependencies
- Calls from the application to other components (including external components) recorded via TrackDependency() - for example, calls to REST API, database or a file system
exceptions
- Exceptions thrown by the application runtime, captures both server side and client-side (browsers) exceptions
pageViews
- Data about each website view with browser information
performanceCounters
- Performance measurements from the compute resources supporting the application, for example, Windows performance counters
requests
- Requests received by your application. For example, a separate request record is logged for each HTTP request that your web app receives
traces
- Detailed logs (traces) emitted through application code/logging frameworks recorded via TrackTrace()

This page from the Microsoft's documentation also holds some information about the data model used by Azure Application Insights.

Plug-in Tracing via Trace

The normal way of having some logs from your plug-in developments is by using Trace Logs. When writing a C# class that implements the IPlugin interface, you can obtain a ITracingService object. This object allows you to write Trace Logs and then consult them directly from Dynamics 365!

Let's build such plug-in. Based on Microsoft's documentation, the following code has been created:

using System;
using System.ServiceModel;
using Microsoft.Xrm.Sdk;

namespace DynamicsChroniclesPlugins
{
    public class FollowUpMeetingPlugin : IPlugin
    {
        public void Execute(IServiceProvider serviceProvider)
        {
            // Obtain the tracing service
            ITracingService tracingService = (ITracingService)serviceProvider.GetService(typeof(ITracingService));

            // Obtain the execution context from the service provider.  
            IPluginExecutionContext context = (IPluginExecutionContext) serviceProvider.GetService(typeof(IPluginExecutionContext));

            // The InputParameters collection contains all the data passed in the message request.  
            if (context.InputParameters.Contains("Target") && context.InputParameters["Target"] is Entity)
            {
                // Obtain the target entity from the input parameters.  
                Entity entity = (Entity)context.InputParameters["Target"];

                // Obtain the organization service reference which you will need for web service calls.  
                IOrganizationServiceFactory serviceFactory = (IOrganizationServiceFactory)serviceProvider.GetService(typeof(IOrganizationServiceFactory));
                IOrganizationService service = serviceFactory.CreateOrganizationService(context.UserId);

                try
                {
                    tracingService.Trace("FollowupPlugin: Begin the plugin.");

                    // Create a task activity to follow up with the account customer in 7 days. 
                    Entity followup = new Entity("task");

                    tracingService.Trace("FollowupPlugin: Building Task record...");
                    followup["subject"] = "Send e-mail to the new customer.";
                    followup["description"] = "Follow up with the customer. Check if there are any new issues that need resolution.";
                    followup["scheduledstart"] = DateTime.Now.AddDays(7);
                    followup["scheduledend"] = DateTime.Now.AddDays(7);
                    followup["category"] = context.PrimaryEntityName;
                    followup["regardingobjectid"] = new EntityReference(entity.LogicalName, entity.Id);


                    // Create the task in Microsoft Dynamics CRM.
                    tracingService.Trace("FollowupPlugin: Creating the task activity.");
                    Guid taskId = service.Create(followup);
                    tracingService.Trace($"FollowupPlugin: Task created with GUID = {taskId}");
                }

                catch (FaultException<OrganizationServiceFault> ex)
                {
                    throw new InvalidPluginExecutionException("An error occurred in FollowUpPlugin.", ex);
                }

                catch (Exception ex)
                {
                    tracingService.Trace("FollowUpPlugin: {0}", ex.ToString());
                    throw;
                }
            }
        }
    }
}

The plug-in is very simple an only creates a Task for a contact, seven days from now. Then, it has been registered with a step on the Contact entity, with the Update event, and with a filtering attribute.

The last configuration is to enable the plug-in trace logs enabled for all events.

And voilà! Now when a Contact record is updated, the complete Trace Log can be find in Dynamics:

Under Settings > Customization > Plug-In Trace Log
Select a record with the name as in your plug-in assembly
Search for the Message Block field

Differences between ILogger and ITracing Service

The following tables comes directly from Microsoft's documentation. It is quite interesting because it states the core differences between those different approached of logs.

Criteria	ILogger for Application Insights	ITracingService Trace for Plug-in Trace Logs
Intended use	Capture telemetry over time for analysis and debugging.	While developing and debugging plug-ins
How long data is stored	According to your Application Insights data retention period, which is 90 days by default	24 hours
Available	Only for organizations that have subscribed to Application Insights integration.	For any organization when Plug-in tracing is enabled.
Amount of data	Each log message can pass a String value.	Only 10kb of text can be written for each plug-in execution. Any more will be truncated.
Available in runtime errors	No	Available in model-driven app client errors and as annotations in Web API.

One key difference is the retention period. Traditional plug-in trace logs are only available for 24 hours. That can be painful when an user experiences a bug, but only brings it up too late or cannot reproduce it. That one of the huge advantage of using telemetry via Application Insights.

The key advantage of the ILogger is to integrate the traditional plug-ins logs within a much bigger scope of telemetry information. In addition to the logs written by the developer, Application Insights also stores when was the code triggered, how long did it run and give some information about external HTTP requests.

Please note that you don't need to choose one or other: plug-in tracing logs and telemetry can work together! For example, if you're building an ISV product, some clients my have Application Insights enabled, some other may not. By having both options, everyone with happy :)

Application Insights Tools

The Application Insights Azure resource tracks both client and server side actions. For example, if you go on to the Performance section of the Investigate pane, you can see both server et browser related operations.

There are a lot of information possible, from the performance to the number of request.

Among all other tools offered by Azure Monitor, there is also the Failures section.

Reference

Dataverse Plugin telemetry with Azure Application Insights

Image

/sites/default/files/2021-08/title_0.png

Azure DevOps for Power Platform : Automation for Build, Deployment, Publish, Export, GIT, Variables

Nguyen Van Hao — Tue, 01 Jun 2021 04:57:54 +0000

Azure DevOps for Power Platform : Automation for Build, Deployment, Publish, Export, GIT, Variables Nguyen Van Hao Tue, 06/01/2021 - 06:57

Body

Azure DevOps for Power Platform:

Azure DevOps is a Microsoft product that provides version control, reporting, requirements management, project management, automated builds, testing, and release management capabilities. It covers the entire application lifecycle and enables DevOps capabilities

The article Azure DevOps for Dataverse using Power Platform Build Tools shows you a basic usage of Azure Pipeline to export and import solution

In this article, we'll implement some advanced setup for Azure Pipeline in the build steps

We'll complete the following steps:

Implement basic setup for Azure Pipeline
Publish CRM customizations
Export unmanaged solution (.zip file)
Unpack the unmanaged solution to a folder
Commit changes of the unmanaged solution (a folder after unpacking) to git
- 5.1 Update permission configuration
- 5.2 Create Window command task to commit solution
Using variables and variable group
Conclusion

1. Implement basic setup for Azure Pipeline

To follow the rest of this article, you need to follow Azure DevOps for Dataverse using Power Platform Build Tools to implement the basic setup.

2. Publish CRM customizations

The power platform extension support publishes customization before exporting.

To do this, add a "Power Platform Publish Customizations" to the build pipeline. You need to specify a connection to the CRM DEV environment

3. Export unmanaged solution

You can also export solution as unmanaged in Power Platform Build Tools

Note: Step 4 and Step 5 will use data from this step.

4. Unpack a managed solution

In Step 3, we already export the unmanaged solution, it's a .zip file

We can use Power Platform Build Tools to unpack (unzip) solution into a folder

Parameters:

Select the unmanaged solution from the build at Step 3
- $(Build.ArtifactStagingDirectory): this is a predefined variable of Azure DevOps
Select the folder name to contain the unpack solution
- $(Build.SourcesDirectory): this is a predefined variable of Azure DevOps which points to the git repository root folder
- Make sure the folder "solutions" exits in your repo
The type of solution is Unmanaged

5. Commit changes of the unmanaged solution (a folder after unpacking) to git

It's helpful to commit the changes to Git to track all changes that go to the PROD environment.

To do this, we can create a Window command to commit changes in folder "solutions".

5.1 Update permission configuration

Open Project Settings at the left-bottom of Azure DevOps

Set "Allow" for permission "Contribute" for the current user or group

Allow scripts to access the OAuth token

If you do not update the configuration correctly, we'll get these error when commit code using Azure Pipeline

Fatal: Cannot prompt because user interactivity has been disabled.
You need the Git 'GenericContribute' permission to perform this action

5.2 Create Window command task to commit solution

After updating the configuration in 4.1, you need to create a command-line task to commit code (the unmanaged solution) into GIT to track change

To do this, create a new "Command line" task

"master": the name of the branch that you want to commit to
"solution": the folder that contains data to commit. See detail in Step 4

6. Using variables and variable group

When you configure for the Azure DevOps task, you can use variables for each config parameter instead of hard code.

There are 3 kinds of variables in Azure DevOps

Pipeline variables: custom variable for this pipeline. You can create as many as you want.
Predefined variables: predefined variables of Azure Pipeline. For example "$(Build.SourcesDirectory)"
Variable groups: Use a variable group to store values that you want to control and make available across multiple pipelines. See detail in "6. Using variable groups from Azure Key-Val"

6.1 Using variables

To access variable, open tab "variables"

Now, let's try to create a new variable "dev.solution_name" and replace the hard code solution name in task "Power Platform Export Unmanaged Solution"

6.2 Using variable groups

Open variable group library

Create 2 sample variables

Link variable group to pipeline

Select your variable group

Now, you can use $(key1), $(key2) in your pipeline configuration.

6.3 Link Azure Key-Val to the variable group

You can also create variable group from Azure Key-Val

To do this you need to register a Azure subscription and setup your Azure Key-Val, then link it to Azure DevOps

7. Conclusion

This article is the advanced setup for Azure DevOps, you can see the basic setup here Azure DevOps for Dataverse using Power Platform Build Tools
Azure DevOps Pipeline is powerful, you can do most CI/CD tasks using it
Using variable is a best practice, you can define a value that you can then use in your pipeline
Azure Key-Val increase security and control over keys and passwords, you should use it when possible

Azure DevOps for Power Platform

Image

/sites/default/files/2021-06/index-hero.jpg

Dataverse PowerDataOps Introduction

Amaury Veron — Wed, 26 May 2021 14:40:18 +0000

Dataverse PowerDataOps Introduction Amaury Veron Wed, 05/26/2021 - 16:40

Body

Dataverse PowerDataOps

Let me introduce you a PowerShell module for Dataverse: PowerDataOps. PowerShell has many benefits: you can edit and run script on Windows servers, execute .Net code, use it easily in many tools, ...

You can find the GitHub for the project here: https://github.com/AymericM78/PowerDataOps. As you will quickly see, the module is very well documented.

This module provides various features to:

Execute CRUD operations on Dataverse data
Work with solutions
Administrate/manage the environment
Perform DevOps operations

You can find the exhaustive list of commands in this file of the GitHub.

The module is built on top of Microsoft PowerShell modules:

It also uses some C# libraries such as Microsoft.Xrm.Sdk.Entity.

This module allows to use very easily the features described above.

To explore and discover the module, I would advise you to start by having a look at these two great samples:

Working with data.md
- In just a few lines, performs the following operations:
  - Connect to Microsoft Dataverse
  - Create a record
  - Retrieve all records and output in a gridview
  - Update a record
  - Retrieve a specific record
  - Delete record
Working with solutions.md
- In just a few lines, performs the following operations:
  - Connect to Microsoft Dataverse
  - Retrieve a solution record (row) with its uniquename
  - Increase version number
  - Export in managed

I hope you will enjoy this module. Don't hesitate to go in the GitHub and explore it more!

Dataverse PowerDataOps

Dynamics 365 Impersonate user using browser extension

Vo Sy Lien Thanh — Mon, 17 May 2021 04:08:04 +0000

Dynamics 365 Impersonate user using browser extension Vo Sy Lien Thanh Mon, 05/17/2021 - 06:08

Body

Dynamics 365 Impersonate user using browser

In Dynamics 365 online, when we need to peform the tests for the security model, It could be a nightmare since we need to login and logout many times and use many browser with private mode and normal mode. Then I found to ease a lot these test sessions thanks to a chrome Extension named "ModHeader". This article will share how to deal with impersonation using webapi and using Chrome extension.

1/ Impersonate using Web API

First of all, I started to figure out how to impersonate another person, I found a document from Microsoft with impersonate detail for WebAPI. You can find it here.

The main point of above documentation is that your running account must have "prvActOnBehalfOfAnotherUser" privilege. And in the header of request need to impersonate must have one of these two params:

CallerObjectId: Azure Active Directory (AAD) object id
MSCRMCallerID: system user id

However, Microsoft prefers to use CallerObjectId instead of MSCRMCallerID. Therefore, in this article, I will show you only the method for CallerObjectId (Actually, you can do the same for MSCRMCallerID. You can try it by yourself! ;))

2/ Add the security privilege to enable Impersonate

As I mention in the previous section, you need to ensure that your account has "prvActOnBehalfOfAnotherUser" privilege.
To enable this privilege, you can open your security role and navigate to tab "Business Management", find "Act on Behalf of Another User" in the last section check enable it

3/ Get Azure AD Object ID of the user:

In Dynamics 365 online, each system user has an "Azure AD Object ID" (azureactivedirectoryobjectid). We need to get user's "Azure AD Object ID" and add it to the header of the request to act on behalf of that user.

The simplest way to get "Azure AD Object ID" is used advanced find and add "Azure AD Object ID" column to the result grid.

You can copy ID and use it in some step behind.

4/ Install ModHeader:

You can find ModHeader extension here or you can open Chrome/Firefox web store and search for ModHeader.

The purpose of this extension is to add some custom params to the header for each request to impersonate.

5/ Use ModHeader:

You can change the name of profile. I usually use email of the account to identify which user I'm using.
Name of header param: CallerObjectId
Value of header param: Azure AD Object ID value

After you finish the setting, you need to refresh the page to see Dynamics 365 in the context of the user you are impersonating.

6/ Stop impersonating:

When you finish troubleshooting the security model, you need to stop impersonating. You have many ways to do it:

Disable ModHeader extension
Remove CallerObjectId
Pause ModHeader extension:

Hope this article will help you save your time when you need to troubleshoot or test security model on Dynamics 365 online!