Knowing When to Ask for Help

One of the skills everyone needs to have is asking for help. Whether that’s in our work, our education, or our personal lives, we all need help from time to time. We are focused on work here, but this same basic rules apply in all aspects of our lives.

The right time to ask for help is, like so many things, a balancing act. Struggling through a complex issue can be a great way to learn something new. But often we can short cut that learning by simply asking the right questions at the right time.

On the flip side, if we ask too early not only do we risk missing a chance to build deep understanding, we also risk frustrating colleagues by asking them to do our job.

One short cut for when you need to ask for help is if another team member asks if you have already asked. Generally, I want to have called for support before my PM suggests it. By then they are frustrated that I haven’t already solved whatever the issue is solo.

Signs You Might Need Help

Given my current role and skill set, I’m often the person who gets called when a project goes sideways. That means I see a lot of places where someone didn’t call for help until they were in crisis. While that’s going to happen to us all from time to time, it’s better to call for help when the problem is small. If you want until the project starts burning down around you, it’s way too late.

You might need help if:

  • You have absolutely no idea what to do next.
  • You are about to re-design a large portion of your project to get around a challenge.
  • You have spent more than a day pounding on a problem without success.
  • You are avoiding working on a task, because you don’t know how to get started.
  • You are about to use a mode/tool/technique that everyone says is a bad idea.
    • In Salesforce that can mean things like:
      • loading data in serial mode
      • setting batch size to 1
      • using view all data in your tests.
    • In Drupal that can mean things like:
      • hacking a module
      • loading data in the theme layer
      • writing raw SQL queries

What to do Before Asking for Help

As I said before, asking for help is a balance: you can wait too long, or you can ask too soon. The real trick is hitting the sweet spot.

There are several things you should always do before taking another person’s time.

  • Google It! I kinda can’t believe I have to say that, but not everyone bothers.
  • Make sure you can explain the question clearly. If you don’t know where you got stuck, how can I help you get unstuck? And thinking it through might make the answer obvious.
  • Develop a theory. When asking for help, it can be useful to pose a theory about an approach. Even if you’re wrong it may help me understand your thinking.
  • Try a few things. Experimenting with what’s going wrong can help you formulate your question, and may help me short cut my research if you have eliminated obvious issues.
  • Explain the problem to your dog, cat, rabbit, stuff animal, etc. As someone who spends time being a human rubber duck, I can often tell when someone tried to explain it once already.

Where/Who to Ask for Help

For me, the hardest part is knowing who to ask.

As a consultant I try to avoid asking questions in places clients may see it. Our clients pay for us to be experts, they do not want to see us asking questions in public – particularly if the question has a simple answer.

As a Salesforce MVP, one of my favorite perks is the MVP Slack channel, where we ask each other questions that run the full range of complexity. While access to a community that hard to access, and that advanced, is a privilege there are other ways to find similar groups like your local user group.

I love having a good internal network of people to ask for help. Most of the places I have worked at as a consultant have had some kind of information place to ask questions and help each other out. If you work in a consultancy find or create such a back channel.

If concerns about being seen by clients isn’t relevant to you, check out this list of 7 Salesforce Communities to Join recommended by Salesforce Ben.

Help Build a Helpful Community

The final thing to know about asking for help, is that it’s important to offer help as well. A good question can be valuable to someone else who has the same issue in the future. A good answer is helpful to both the person who asked the question and the person who looks again in the future.

But offering answers, even if not perfect answers, is a great way to learn and encourage others to seek help. Any time I post a question on Stack Exchange, I try to hunt around for one or two to answer as well. That both allows me to pay it forward, it also helps encourage the tone that people can be experts in one thing while still needing help in another.

Smart people need help, and should be comfortable asking for it.

Writing for Developers and Consultants: Know your Documents Types

When I started this series on writing for developers and consultants, I thought of this piece first, but I couldn’t get the ideas to come together. Sometimes it takes a few tries.

Anytime you write something, you should be aware of what kind of document you’re writing and who is it for. The definition of “document” in this context is very broad it could be: an email, letter, solution design, chat messages, blog post, test script, work of fiction, book, poem, presentation, marketing slide deck, scope of work, and so on: anything that involves writing words.

For example, this is a blog post, about writing, meant for people who are developers or technology consultants who may have been told good writing isn’t important.

Common Work Documents

I’m going to put aside poems, novels, and other personal writing forms to focus here on work related writing. Developers need to be able to describe their work to other people. We also need to communicate about what is happening on a project, support one another, and ask for help. We do all these things with words, and often in writing.

In my career I’ve written a wide variety of documents as part of my work. A partial list (because I doubt I can think of everything to create a complete list) includes:

  • Emails
    • to my boss
    • to colleagues or friends
    • to direct reports
    • to clients
    • to large mailing lists
  • Solution Design Documents
  • Scopes of Work
  • Contracts
  • Invoices
  • Test Scripts
  • Conference Talks
  • Research Reports
  • Chat Messages

Some require more detail and longer prose than others. Some are expected to be polished where others can tolerate typos and mistakes. But each has its own style, structure, audience, and expectations that the writer must meet.

A Document’s Purpose

When you start to wring something, know your purpose in writing.

Not all documents are created equal and so understanding your purpose is critical. Are you writing an Solution Design that needs to outline how you plan to solve all the hard problems in a project? Or are you writing an email to your boss asking for a few days off? Is this a research report meant to support an advocacy project or a cover letter for a resume? All of those are important things, but none should be written in the same tone or with the same style.

A Scope of Work (SOW) is a lasting artifact during a projects that sets the bounds of the work you’re going to complete. A sloppy SOW can cost you, or your employer, vast sums of money. A SOW writing purely to defended against those concerns may not express the client’s needs and interests, and result in them refusing to sign.

An email to a client might be a friendly reminder about pending deadlines, or a carefully crafted notes from a contentious meeting. Written well, both could leave you in a better place with your client. Written poorly, both may cause your client to become frustrated with your sloppiness.

If you don’t know why you’re writing something, you are likely to write the wrong thing. At work, if you aren’t sure, ask for guidance.

A Document’s Audience

There is no such thing as a “general audience” you should always have a mental image of who you are writing to, and why.

We all know that it’s important to think about your audience, but we don’t always do this well. In part because determining the audience is sometimes a little complicated.

When your audience is the person or people you are writing to, you need to leverage your understanding of their knowledge, skill set, and project engagement. You want your text to meet them where they are.

Sometimes the audience you care about most isn’t the direct subject of the message, but a 3rd party you know, or suspect, will read the document later. I find this is true particularly in contentious situations.

FOIA Warning

If you work in, for, or with government agencies in the US (and for similar reasons elsewhere as well) – including as a subcontractor – you should understand if your content is subject to a Freedom of Information Act requests. Sometimes your audience isn’t the person you are writing to at all, but the reporter who could read the message 2 years from now after they get copies of everything related to your project. In those settings, don’t put anything in writing you don’t want on the front page of a major newspaper.

But FOIA can also be a blessing for a developer who knows a bad decision is being made. Carefully worded expressions of concern, and requests for written confirmation of next steps, can trigger FIOA-cautious readers to recognize they need to follow your advice.

Finding the Right Level of Technical Detail

One of the hardest things for developers, and other people with lots of technical knowledge, to do well is communicate clearly about technical minutia. There is a balance to be struck between providing transparency and overwhelming readers with details. Developers have to think about details in our work. We also use field specific jargon that can be confusing to people whose work is in other areas.

Too often we confuse that specialized knowledge of our field, with intelligence. I have watched developers lose their audience in the nuances of small details, and come away announcing their audience was a bunch of idiots. Early in my career I was guilty of this as well. Assume you’re audience is as smart as you; they just know different stuff.

When you make that assumption you can avoid talking down to people, and start to work on finding their level.

The right level of technical detail will also vary by document type. When I’m exchanging emails with a client’s in-house developer we go deep into the weeds often. When I’m writing a SOW, the technical detail is nearly absent as we are defining functionality and purpose, not the exacting detail of how that functionality will be delivered.

The more you can be in conversation with the people you’re working with about their background, the easier it will be to find the right level of detail to explain yourself clearly.

Summation

Hopefully by now it’s clear, this is an overview of approach, not detailed guidance. In a future post I plan to write about some of these specific documents types, and how to write them. Hopefully this overview gives you ideas and things to think about as you work on your next document.

As I said in my first post on this topic, communications skills for developers and consultants is an enormous topic. The plan for this series is evolving as I go. If you have suggestions or requests feel free to leave me a message.

Tool Building Mindset

Earlier this month, at Mid Atlantic Dreamin’ in Philadelphia, I gave a talk titled Software Super Heroes: Building the tools you wish you had. My goal with the talk was to convince people that should, can, and in fact do, built tools for themselves. If you work with technology, and your job involves repetitive tasks the same applies to you too.

What do I mean by “Tool” and “Tool builder’s mindset”?

I like to use a very expansive definition of tool: “A tool is anything that makes a task easier which would be repetitive, hard, or impossible without it.” In that sense just about anything you make that simplifies you work can be considered a tool: a project estimation spread sheet, a good set of directions for a complex task, a flow for a Salesforce admin, a piece of code to normalize a large collection of files, and more.

My intention with that expansive view is to help encourage people to take on a tool builder’s mindset.

To be a digital tool builder does not require knowing how to write complex software, it just requires you to do what you already do now, but with intention. When we use a broad definition of tools, it’s easier to see ourselves as tool builders, even if we’re just talking about a spreadsheet or a Salesforce flow meant to handle an administrator’s daily tasks. When we see ourselves as tool builders we are more likely to make something worth using more than once.

Why does this matter?

When we approach problems with a tool building mind set, instead of insurmountable challenges caused by gaps in our tooling, we see opportunity to create something new to make the impossible possible. Instead of facing hours of boring repetitive tasks, we have chance to build a more interesting special purpose solution.

Fight the Tool Building Excuses

There are several excuses I commonly hear from people when I encourage them to build their own tools. They range from concerns about not having the right skills, to assuming someone else already built that tool or that the time required isn’t worth the effort.

My general response to these concerns is that while people should indeed look around for tools that already solve their problem, and that some problems are very hard to solve completely, if you start to chip away at a complex problem you often will find that you can create tools that are good enough to save you time and effort.

Don’t try to build the perfect tool that solves all possible edge cases on your first go. Create a tool that takes out some annoying and repetitive task. Then create a tool that solves for another task, or builds on your first time. Chip away.

I often tell developers who are early in their career that I should never see them doing rote repetitive tasks for hours on end. Instead once they understand how a repetitive task is done, they should start thinking about how to build a tool to take over. But that’s not just advice for developers: we invented computers to do repetitive asks (calculating artillery firing tables and cracking codes), let them do that.

Pick Your Tool Building Path

When you set out to create a tool you have two main options: use something you already know, or use tool building as a chance to learn something new. I’ve used tool building as was to teach myself new features of Excel, Google Sheets, Salesforce Flows, Git, and several programming languages. This can be a great way to learn how to use the tool that’s just right for the job. But learning a new tool or technique takes time, and if you have a deadline you may need to move faster.

Personally I try to take both paths from time to time. I use things I know when: they are exactly the right tool, I am under time pressure, or I want to keep my skills sharp. I will take the time to learn something new when: it’s something I need to learn anyway, I am building on my own time, or it’s exactly the right tool for what I need to do.

Neither path is correct 100% of the time. By using them both I am able to create the tools I need, and broaden my skills over time.

Just Start Building

The next time you’re faced with a task that is repetitive or hard with the tools you have: create yourself something new. Don’t get hung up on being perfect, just create something that’s better than what you have at the start.

Then save your tool to use again later. Share it with colleagues, friends, or as an open source project.

When in doubt, just start building.

Writing for Developers and Consultants: Use of Language

Words matter.
Grammar matters.

How we use language affects how our audience perceives us. In school teachers and professors taught many of us formal – and strict – rules for writing. Those rules are useful to know, but my point is not that you need to follow them strictly.

You need to control the language you use with intention, and create the impression you want your audience to have.

When we write at work, we are a reflection of ourselves, our team, and our companies. We want all those things to look good to our clients, customers, leadership – our audience. That is not the same goal as learning all the rules of Strunk and White. It’s about choosing the right words and structures to meeting our audience’s expectations.

To do this well, you need to understand the patterns and rules people think of as formal writing, and when to use idioms or patterns that break those rules. Sometimes it is better to follow the rules; sometimes it’s better to break them.

It’s a matter of fashion, pure and simple. People do need to be taught what the socially acceptable forms are. But what we should teach is not that the good way is logical and the way that you’re comfortable doing it is illogical. It should just be, here is the natural way, then there’s some things that you’re supposed to do in public because that’s the way it is, whether it’s fair or not.

John McWhorter

Strategic Use of Passive Voice

In college several professors demanded I never use the passive voice in my formal papers. Microsoft Word of the day backed up that assertion by flagging every passive sentence as a grammar error. In those papers they were right, since they controlled the grading standard, but those are not the rules I follow today.

Shortly after graduation my wife and I discovered the power of the passive voice when our rabbit chewed through a power cable of our brand new printer. When I called support I said: “The power cable is frayed.” They didn’t ask how the cable became frayed, they just assumed it had arrived that way and shipped us a new one. Perhaps not our most ethical moment, but a useful one in understanding the power of breaking the formal rules we’d been taught.

A simple definition of the passive voice is:

A passive construction occurs when you make the object of an action into the subject of a sentence. That is, whoever or whatever is performing the action is not the grammatical subject of the sentence.

UNC Writing Center

The classic joke is “Why was the road crossed by the chicken?” where we have clearly reversed the subject and the object of action. More commonly we leave the true actor, the person who should be the subject of the sentence, out of the sentence entirely. There are lots of discussions on the passive voice around, because it is an important concept to understand. But banning passive sentences is the wrong approach.

Passive construction makes sentences weak and frequently unclear. Without editing I will often write too many passive sentences in a post. But sometimes the weakness of passive construction helps to drive my point home.

In writing with clients over use of active voice can become accusatory. Think about the difference between “The data set provided has many errors.” vs “You provided me a data set with many errors.”

The first gives the reader room to blame a file, a process, or another cause other than themselves (which may be correct). The second points a finger. When I am trying to resolve a problem caused by bad sample data, pointing fingers, placing blame is not helpful. Sometimes we need to be clear, and identify actors explicitly. Sometimes we want to indirect to avoid creating unneeded tension.

Other Rules to Follow Strategically

In school our teachers taught many of us that paragraphs should always have three, or more, sentences that mirror the three sections of a paper: introduction, body, conclusion.

But a sentence by itself stands out and draws attention.

If every sentence were by itself they would stop standing out, so mixing emphasis and longer structures is still a good idea. The readability checker I use for my blog dislikes long sentences, preferring short choppy structures. Short punchy writing is easy to skim, but exhausting to read. Overly long sentences are hard to follow and may reveal incomplete thinking because they contain too many ideas. It’s a balance to be used carefully.

Punctuation is also taught as a series of strict rules. However use of commas, semicolons, periods, dashes, and so on are also matter of personal style. If you know the purpose of a mark you can decide when you want to use which to add emphasis and clarity to your text.

Starting sentences with conjunctions was most gracefully debated in Finding Forester (movie staring Sean Connery as an aging writer teaching a young African American boy how to become a writer):

Forrester: Paragraph three starts…with a conjunction, “and.” You should never start a sentence with a conjunction.

Jamal: Sure you can.

Forrester: No, it’s a firm rule.

Jamal: No, it was a firm rule. Sometimes using a conjunction at the start of a sentence makes it stand out. And that may be what the writer’s trying to do.

Forrester: And what is the risk?

Jamal: Well the risk is doing it too much. It’s a distraction. And it could give your piece a run-on feeling. But for the most part, the rule on using “and” or “but” at the start of a sentence is pretty shaky. Even though it’s still taught by too many professors. Some of the best writers have ignored that rule for years, including you.

More or less any rule or pattern you have been taught, you should explore when to follow and when to deviate.

Practice Until Your Comfortable

I know of no magic way to become comfortable using language carefully, except by doing it. Be thoughtful when you write. Edit carefully at every chance. Ask for feedback when available. And write in as many contexts as you can justify.

My whole point in keeping a blog, even well passed the era of independent tech blogging (and with limited care for the theme being used, SEO, or monetization) is to force myself to write regularly. I write different types of posts with different styles and attention to care. Some of that is months I’m being lazy, and some of that is very intentional.

I first wrote for this blog when I had a job that actively discouraged me from communicating well – I was told time and again that developers cannot write clearly. That’s crap: it was then, and it is now. So I started writing at least monthly to keep myself in practice.

Your practice does not need to be a blog. Find yourself some form of routine writing, and play with your style over time. Just a few suggestions of things to try:

  • Letters to friends and loved ones (on actual paper and with a stamp)
  • Short Stories or other fiction writing
  • Contribute project documentation to an open source project
  • Essays for Medium (more or less blogging with less commitment)
  • Participate in an online writing challenge
  • Journaling
  • Take a writing course

Still More to Come

As I said in my first post on this topic, communications skills for developers and consultants is an enormous topic. I have not yet planned the series, but you should expect more to come.

Birthday Baking

Every year I bake something for myself on my birthday. It’s a pair of things my mother accidentally taught me.

While for much of my life my mother did not like to talk about her own birthday, she loved to celebrate mine, my sister’s, her grandchildren’s, all were cause for her to bake a cake, or some other marker, to share with our friends. But my mother never ate those cakes: she was allergic to wheat.

A picture of a picture, featuring my mother and a few friends watching me as I'm about to cut a birthday cake.
That’s me in read, about to serve cake and ice cream at a birthday party.

She baked a lot at other times as well: wedding cakes for family and friends, pies and cheese cakes for holidays, mountains of cookies for the annual Christmas open house, and snacks for her students just because they might need a pick me up. Through all that baking, mom taught me that baking was something you did for other people. It wasn’t her intention, but it was what I learned. One of my first memories of baking myself was making an Angel Food Cake for my grandfather.

An angle food cake recipe, with Aaron 3/92 written at the top.

And so I only bake for myself once a year: to celebrate my birthday. This year’s cake was adapted from one in The Great British Baking Show: Love to Bake (which needs adaption as it’s written for a British audience and so has different flour, sugar, measurements, and nomenclature from US-centric cookbooks.)

Chocolate cake, with a slice missing. White cream filing showing between the layers.
My 2024 cake with my first piece missing
Chocolate cake sitting on a white cake holder base.
The 2024 cake still intact
Yellow cake, with lemon curd between layers.
A lemon cake from a couple years ago.
A pastry with a cream filling. About 1/2 consumed.
A custard filled pastry.
Apple pie-like pastry, with a large wedge already missing.
A apple filled treat I made for myself one year.

I’ll get back to my series on writing for developers and technical consultants soon. But the next couple of pieces are still coming together. Sometimes editing take awhile.

Writing for Developers and Consultants: Editing

The most important thing you need to have to be a successful consultant is excellent communications skills . If your client doesn’t understand and trust what you tell them, it doesn’t matter what you suggest, create, build, or deliver.

I have been helping a few colleagues lately work on their client communication skills and figured I should write down my suggestions for how developers can communicate well with clients.

Everyone on a project team needs to be able to communicate what they need, what they are doing, and how their piece of the project works. They also need to know how to address their own errors and mistakes so the project moves forward and the client trusts that errors will be resolved.

As I sat down to write this piece I realized the topic is huge; far too large for one post. I’ve thought about the importance of good writing, editing, grammar, the kinds of email messages we send, documents we write, and presentations we give. 

All of those probably justify at least one post, so I’m starting here: editing.

Why Editing is Important

Editing probably feels like an odd place to start a series on writing, but it is a thing you can start to do now that will help you write more effectively. If you have a college degree you probably got into the habit of writing papers fast, and turning them in at the deadline. But you almost certainly didn’t do enough editing of those papers (I’m married to a college professor, and I talk to college professors a lot: No. You. Didn’t.). If you don’t have a college degree, it’s also unlikely many people have sat down and talked to you about the importance of editing. If you’ve come to consulting from a writing-heavy background you’ve figured out by now the value of editing – this post isn’t really for you, so please come back for later installments.

Editing is how you take poor, or merely acceptable writing, and make it clear and compelling. 

Good editing is about more than basic mechanics, it’s about testing that your idea or explanation makes sense. Editing lets you clean up awkward, but technically correct, expressions. And it gives you a chance to reflect on how your audience will read the material. As you edit, see if each sentence makes sense to them the same way it does to you.

Editors Are A Luxury

A good editor is worth their weight in gold – sometimes more. Well known authors have been known to change publishing houses to follow a good editor. They are also not part of your typical consulting team. For consultants, having an editor is a rare luxury. 

For large documents, good teams will make sure one or two people give completed drafts an editing pass. Often getting support from a marketing department to edit a conference talk is possible. Another person reading an email and offering edits happens exceptionally rarely. It’s the same with internal presentations. No one will edit your chat messages except you.

So, while an outside editor is an invaluable tool, a consultant should not expect to have access to one.

Be Your Own Editor

That means you have to be your own editor. Editing your own writing is hard. If you could see the mistake the first time, you would have fixed it before you write the next sentence (those kinds of fixes are not the same as editing, that’s “fixing typos” or “not being lazy”).

There are three main tricks I use when being my own editor:

  • Put the document aside between writing and editing – the longer the better
  • Reading it aloud to yourself (yes really, out loud)
  • Edit on paper (use one of those printer things)

I wrote the first draft of this while sitting in the Atlanta airport. I revisited and edited a printed draft a few days later (my wife also took a pass – see above). And again after I put it into my blog. It’ll probably get a few tweaks after it’s posted.

Don’t Let Edits Be Personal

Writing, even technical writing, is very personal; editing should not be. 

A good edit may be brutal to the original material. If you get too attached to your first draft, you will be tempted to skip asking for an edit or doing an editing pass yourself. That’s part of why I put a document aside for a bit between writing and editing, it gives me space to see the flaws in my words and ideas.

Several years ago I was editing a long report for AFSC, and the first draft was far too long, too detailed, had too many tangents, and didn’t focus on the intended audience. I attacked it. She and I spoke several times to make sure she understood my goal was to help make her was succeed. Through aggressive editing we made sure her great work achieved her intended goal: stopping the unchecked growth of private prisons.

I deleted entire sections, rewrote the summary (which someone else then edited for me). I pulled in favors, got my wife to edit drafts. We debated chapter order, technical details, and sentence after sentence. What started at 100 pages, ended up being 20 page report with an 8 page summary that journalists and politicians cited as they debated proposed private prison expansion. Because of that work, and follow on efforts by people using it as a model, private prison growth has slowed substantially (if that’s not a topic you know about, trust me that’s a good thing). The author and I became friends and worked together well on follow up projects and efforts. She appreciated the value of the edits even if they were painful at first.

Edit Aggressively

Whether your editing for yourself, or for someone else, dig deep and edit aggressively. If you haven’t done much editing you want to think about every word, every piece of punctuation, every detail. As you get more practice you’ll find short cuts and learn your own weaknesses (or your team’s).  I use too many words, and write long sentences – I attack those. Shorter is better.

Using traditional grammar tools can be helpful, particularly if you are unsure about the basic grammar rules. But if you use tools like Grammarly or ChatGPT to rewrite things make sure you think carefully about if, and why, you like that version better. Used well Generative AIs are a useful tool, but if you lean on those tools too much you don’t actually get better yourself.

More to Come

As I said at the top, communications skills for developers and consultants is an enormous topic. I have not yet planned the series, but you should expect more to come (and I’ll edit this section to add references).

Salesforce Github Actions

I started this post back in November, but got rather distracted last month.

I have been wanting to setup a CI pipeline for Salesforce scratch orgs for a long time. The documentation out there isn’t great, and it’s taken me awhile to get around to powering through the details. For testing that my scratch org configurations for Education Cloud and Nonprofit Cloud to work properly I finally bit the bullet and setup a Github action.

What you’ll need

  1. An SFDX-style project on Github that you want to setup for automated testing that includes a Salesforce scratch org.
  2. An org with Devhub enabled. I recommend using a production org for this, not a developer org because of the higher scratch org limits. There are no good reasons not to enable Devhub.
  3. A working install of sf cli with a connection to that devhub-enabled org.

Setup Github Secret

For your action to work it will need to use your connection to your Salesforce Devhub. Salesforce CLI stores the needed key on your local machine, which includes tokens to open the org. You will need to put those keys into a Github secret for proper security.

Note: Should that key ever be compromised, they attacker will be able to open the org as the same user. I suggest you consider using a dedicated user with either a Free Limited Access License, or another similarly restrictive set of permissions. These users are designed to avoid exposing data if they are compromised (they are also free, which has some upsides). That said, you can get yourself started with any user that can access to Devhub objects and then switch out to something more restricted later.

To get the key from sf cli, run sf org display -o [your devhub alias] --verbose where [your devhub alias] is the name of the connection you selected when setting up the connection. The output of that command will include: “Sfdx Auth Url”, that is the value you need in your Github secret.

  1. In Github, go to your repository’s settings, expand Secrets and Variables from the left menu, and select Actions.
  2. Create a new Repository secret.
  3. Name the secret DEVHUB_SFDX_URL
  4. Copy and Paste the Sfdx Auth Url value into the Secret field.

Create the Github Action

The full details of Github Actions are a deep topic, as are the full capabilities of sf cli for testing. So I’m going to focus on setup of an action the creates your scratch org. From there you can expand your workflow as you need.

In your project root create a .github folder, and workflows folder within if you don’t already have them. In the workflows folder create a file called buildScratch.yml.

Copy the following code (explained below) into your file:

name: test run scratch

# Definition when the workflow should run
on:
  # The workflow will run whenever an event happens on a pull request
  pull_request:
    types: [opened, synchronize]

jobs:
  validate_scratch_deploy:
    runs-on: ubuntu-latest
    steps:
      # Install Salesforce CLI
      - name: "Install Salesforce CLI"
        run: |
          npm install @salesforce/cli --location=global
          nodeInstallPath=$(npm config get prefix)
          echo "$nodeInstallPath/bin" >> $GITHUB_PATH
          sf --version
      # Checkout the code in the pull request
      - name: "Checkout source code"
        uses: actions/checkout@v3
      # Load secret for dev hub
      - name: "Populate auth file with SFDX_URL secret"
        shell: bash
        run: "echo ${{ secrets.DEVHUB_SFDX_URL}} > ./SFDX_URL_STORE.txt"
      - name: "Authenticate with dev hub"
        run: sf org login sfdx-url -f ./SFDX_URL_STORE.txt -a devhub -d
      # Create a scratch org
      - name: "Create scratch org"
        run: "sf org create scratch -d -f config/project-scratch.json -a our-scratch-org --duration-days 1"
      # Deploy Project Manifest
      - name: "Deploy Metadata"
        run: "sf project deploy start --manifest manifest.xml --target-org our-scratch-org"

Action Breakdown

The first few lines are common to all actions, it gets a name, then a set of conditions to trigger the workflow. In this case we’re calling our workflow “test run scratch” and it will run when a Pull Request is opened or gets new commits.

The jobs section is the more interesting bit. We tell Github to create a virtual machine using the most recent version of Ubuntu Linux. Next we install Salesforce CLI onto that virtual machine. Once that’s done, we tell it to checkout our code from Github onto this virtual machine. With all our tools in place we’re ready to start the real work.

The next two steps extract the Github secret we defined earlier into a file on the virtual machine and use that file to authenticate to your Salesforce devhub. All that work gets this temporary virtual machine into the same position as your personal device probably was when we started.

The last two steps create the scratch org based on a hypothetical scratch org configuration file and gives the scratch org a short life of one day. Finally deploy your project’s manifest-tracked metadata. This last step could use any version of the deploy command. It could also be added by more steps to deploy other metadata, run tests, or anything else you want to have happen.

Closing Thoughts

Building from this point you can do all kinds of things. For example, if you don’t need to gain access to the org itself, you can include a step to delete the scratch org when you’re done to help avoid the active scratch limit.

You’ll want to pay attention to the limits of your org to see if you need to tweak the triggers of your workflow. If you aren’t using a production org you scratch limit is only 6 per day, and three at a time; I blew through that testing the steps in this article. Even larger orgs only have 200 per day. So be thoughtful about when your action runs, and how often you clean up.

As always, if you find a flaw in my solution please let me know. I’m always interested in making these better.

Goodbye Mom

Earlier this month my mother, Maria Crosman, passed away. Her life came to a peaceful end after a long slow decline. My sister, father, and I were at her side at the end.

It’s hard to know what to include when writing a remembrance of Mom. No one post can really cover it all. So I’m not really trying. In addition to being my mother, mom was a minister, teacher, mentor, pastor, gardener, grandmother, and friend. She could be serious, stern, loving, warm, and funny. When the situation called for it, she could switch between those mode quickly.

At 15 she felt called to ministry, and was immediately told by her pastor that ministry wasn’t for women. Since Quakers had encouraged women to be ministers since our founding, Mom promptly began attending Abington Meeting. She went on to earn Masters and then a Doctorate in Ministry and a recorded Friends minister. Along with my father she served as pastor to two churches. Before finally becoming a teacher and sharing her gifts with her students and colleagues. Over the course of her life she found a multitude of ways to let her life speak.

Trying to tell my mother she wasn’t welcome never got a meek response.

For roughly 40 years her life intersected with those of George School‘s students and the larger community there. Her goal was always to find ways to help the students grow into the best versions of themselves.

My first two years as a student there I ended up with a locker about 8 feet from her office door. She was running the student co-op program at the time. Every student at George School performs basic tasks like cleaning classrooms and serving meals, and her job was to issue those assignments. Mom believed deeply in the program, and in making sure privileged students did their share. I know first hand that not all my peers appreciated the value of that form of learning. But we have heard from former students in recent weeks, and found thank you letters from students, colleagues, and even peers at other schools among her papers, all testifying to the mark she left on those around her.

Some of those students who grumbled about early co-op assignments, appreciated her support when they found themselves needing extra love in a hard moment. I recall vividly an evening a friend of mine recognized another friend in crisis, a quick call and mom returned to campus to set in motion interventions that saved the life a student. She could draw lines and set boundaries that teenagers need in one moment, and open her arms for hugs we needed even more in the next.

As we have heard from friends and former colleagues many people have shared memories about her craft projects, and how she taught them to knit, quilt, bake, garden, and other crafts. For Mom those were all forms of connection.

She baked, mostly with flour that she could not eat, so others would have a treat to enjoy. Our family households are filled with wedding quilts. Knitting projects done during faculty meetings were often presents for others – and the chance to sit and knit with another person was a chance to form or deepen a friendship.

Even after she stopped formally leading congregations mom still found ways to do the work of a pastor as well. She loved performing weddings for friends, family, or even strangers. At Silver Bay she served as summer chaplain, and spent time as interim chaplain at Adirondack Friends Meeting. Never one to stay on the sidelines, she stepped into all kinds of situations from memorial services to medical crises because she saw a need she could fill.

My wife once saw my mother jump into another family’s medical crisis. The doctor was doing a poor job of explaining a scary situation to young parents in an ER. So poorly the father was nearing the point of striking the doctor. Mom inserted herself and managed to calm the father. She validated his fears and reminded him of the importance of stay calm for his child’s sake. She also took the doctor to task for not being clear and supportive and pushed him to help the parents understand. We have no way to know what happened to the child, but in that moment my mother was able to help everyone find next steps.

Having grown up in an often divided family, she sought to find and build connections within the family she built and between us and our community.

Everywhere she went, my mother tried to make friends. Often those friends would find themselves called on the help in crisis. When a student needed more counseling than the school could provide – she called a therapist friend. When a student needed advanced dental work and lacked the means to pay, she called another friend. She’d come home from nearly any place with stories of people she’d met and the interesting things they had taught her.

I miss her. I expect I always will. But I’m happy for the time we had together, and the lessons she was able to teach me. Especially the importance of connections with others.

Salesforce Nonprofit and Education Scratch Orgs

During the recent Open Source Commons sprint in Chicago, I tried to create scratch orgs for nonprofit and education clouds. Despite having some of the best people in the market in the room, including two Salesforce Solution Engineers, two levels of their bosses, and of course Google, I couldn’t figure it out.

As a follow up to a conversation there, Larry Fontillas sent me links to the help docs that contain what I consider partial answers. While I’ve sent feedback to help improve those articles, I am posting my current solution to this challenge.

My Example Scratch Config

On Github I created a repo that contains two scratch org configuration files:

Each is my attempt to create a definition that works for the named cloud. If are new to scratch orgs, I suggest you start with the Trailhead Salesforce DX Quick Start. With a Devhub setup, and connected to your sf cli, you can easily create these scratch orgs from my settings with one of the two following commands:

sf org create scratch -d -f config/nonprofit-cloud.json -a npc-org
sf org create scratch -d -f config/education-cloud.json -a edu-org

Industry Org Scratch Config Breakdown

The two new clouds leve re-usable components Salesforce built, licenses, and deploys across different markets. Salesforce does not currently provide one master switch you must use. Instead you need to know what features to include and tell the Devhub which collection to enable.

That is done in two major parts of the configuration file. In my Nonprofit cloud config file there are several sections, but two are critical: features and settings → IndustriesSettings. The features list includes Salesforce components to enable. In this case I included the nonprofit specific Fundraising, Program Management, and Grantmaking modules, but also OminiStudio, Accounting Subledger, and more because they included with NP Cloud by default. Under Industries Settings you’ll also see I enable Grantmaking and support for Program Management.

The Education Cloud config file has even more detail. That’s because Salesforce as made more available for Education Cloud. The Industries settings section includes more flags as well for the same reason. As I Followed the setup guide for Education Cloud, I further adjusted some of the base-line object permissions.

Here are the features I know you need to include for each cloud:

FeatureNP CloudEdu CloudNotes
AccountingSubledgerGrowthEditionOptionalOptionalStarter Edition is also available
AccountSubledgerUserOptionalOptional
AnalyticsQueryServiceYesOptionalThis is listed in the docs under Fundraising, but I have not yet found a direct description.
AssessmentsYesYes
EducationCloudNoYes (requires a quantity parameter)Main Education Cloud objects, but only a sliver of the features.
EnableSetPasswordInApiYesYesAllows the cli to set the password, you always want this.
FundraisingYesOptional
GrantmakingYesOptional (Rare)
IndustriesActionPlanNoYes
IndustriesSalesExcellenceAddOnYesYesThis is listed in the docs under Fundraising, but I have not yet found a direct description.
IndustriesServiceExcellenceAddOnYesYesThis is listed in the docs under Fundraising, but I have not yet found a direct description.
LightningSchedulerNoYesLightning Scheduler gives you tools to simplify appointment scheduling in Salesforce.
LightningServiceConsoleOptionalYesAllows the Lightning Service Console and access features that help manage cases faster.
MarketingUserYesOptionalProvides access to the Campaigns object.
OmniStudioDesignerYesYesListed in lots of samples, but I have not yet found a direct description. But clearly needed for OmniStudio.
OmniStudioRuntimeYesYesMore for OmniStudio.
OutcomeManagementYesNo
PersonAccountsYesYesWe all love Person Accounts now! Technically this is optional, although the assumption is it your default.
ProgramManagementYesNoEnables the NPC Program and Case management features.
PublicSectorAccessNoYesNot entirely sure about this one, but some of the features for Education Cloud seem to leverage these objects and settings.

Feedback Please!

I have tested these configurations to the degree of seeing that they work as a basic level. But I have not, yet, used them for a serious project. I am confident other people will find details that are missing, or just wrong. Please file an issue on Github or leave me a comment here with suggested changes.

A Salesforce Data Migration Pattern

Loading large amounts of data into Salesforce is a non-trivial exercise. While traditional databases can often be loaded in nearly any order, or with just a few simple considerations for foreign keys, Salesforce’s platform behaviors require several special considerations.

Over the last few years I’ve done a number of large data migrations into Salesforce, and developed a pattern I like to follow. This pattern allows me to load data efficiently at any scale.While the implementation details will vary, you can adapt this pattern to your projects. 

Efficiency matters more the larger your project: for a small project, this is overkill. If you are loading 1,000 Contacts it will probably take you longer to setup my process than just format the file in Excel and load it through Data Loader. But if you need to load 100’s of thousands of records, millions of records, across lots of different objects, this pattern can save hours or even days.

Migration Process Overview

The general concept here, is that you’ll run your migration in two major phases:

  1. Prepare the data in a staging database.
  2. Load the data into Salesforce.
Diagram of a two stage migration, first from the source data into a Salesforce staging database, and then from the staging database into Salesforce.

Salesforce Schema Mirror Staging Database

The key to this process is that staging database in the middle. 

In my experience having a database that is a clone of Salesforce’s schema allows you to fully prepare the data prior to loading. It also gives you a source of truth when handling partially loaded data.

Salesforce is slow to load compared to most traditional databases. By having a staging database you can load fast, gives you a chance to insert steps into your process that are hard in other contexts. These steps allow for testing, speed-enhancements, and error recovery.

Some ETL tools make a staging database easy to build, others do not. If you aren’t sure how to build such a database (or it seems like a huge effort to re-create all those tables), you can use Salesforce2Sql – that’s why I created it. It will clone your Salesforce org’s schema into any of its supported databases.

Testing and Error Recovery

The staging database lets you test for errors after you do your initial conversion; before you load it into Salesforce. You can leverage reporting and scripting engines designed for that database. You can log and error trap during your loading process far more gracefully than the Salesforce APIs support by default.

I often add one more table than just the objects: a logging table. This allows me a place to write the rows from Salesforce error files, and also log the time it takes for each process to run. I can see exactly what errors my process encounters at the record level during testing, and measure the running time.

This database will also give you a place to trace what has, and has not, been loaded into Salesforce. More about how to implement this and drive performance to come.

Transform the Data

Using the tool of your choice, create a process to transform the data from the source data into your staging database. How you do this stage could be a series of posts by itself. For my ideas on a good process for this I suggest my Queries on Queries talk.

Your process will have a mountain of small details – I often describe it as “hard and boring”. Done well this is your best point in the process for testing your work. Test thoroughly! You should run this process so many times you lose count.

Salesforce Migration Keys 

One important detail is that you will want to leave the main record Id field null. Your legacy Id goes into a legacy Id field, but the main Id field should be empty. We’ll use that in the loading stage to determine which records were successfully loaded and which need follow up attention.

Every object you are migrating should have a legacy Id field that links back to your source database. These should generally be text fields, set to be external Ids and unique. These fields will both help with the migration itself, but also the validation process – and should you need to, you will be able to update the data post-migration using those same keys.

To handle references between records use the legacy Ids as the lookup Id values. For example, on a Salesforce Contact there is an AccountId field to reference the parent account. The Account’s legacy Id should be in AccountId. Often this value is already in your foreign key fields so it can be a real time saver in your transformation build. We’ll see in a minute how we use those to resolve to new Salesforce Ids as we load data.

Data Cleaning

This is also the time and place to do whatever data cleansing you plan to do in your process. You can do that work post-launch as well (mostly). I highly recommend this cleansing be automated for large data sets. If you can’t automate it, do it pre-migration in your old system, or post-migration in Salesforce.

Pre-Load Data Validation

Using the staging database your transformed data can be fully validated before you load it.

  • Check your references: Make sure all your lookup fields are populated with valid data. 
  • Check your record counts: Do you have the expected number of records in every table? 
  • Check your critical fields: All data points are created equal, but some are more equal than others. Check those a few extra times.

If you have the time and resources, you can write scripts and other automations to run these tests for you. The more the better.

Loading Salesforce

Finally, all that data you just transform and staged is ready for high volume loading. For each object you run two steps:

  1. Insert the data via the Bulk API (Insert not Upsert!), record the start and end time and all errors in your log table.
  2. Update the records in your staging database to add the new Salesforce Id into the source record’s Id column (the one I told you to leave blank before).

When there are no null values left in the Id column, you have loaded all your data. If there are records that refuse to load, for any reason, you will know because the Id will be null. If you logged the errors you can see why.

You will also use those Ids in later jobs to update the reference Ids. Remember, we put the legacy Id into your reference fields, when you actually load the data you need to replace that legacy Id with the actual Salesforce Id.

When possible you should build these load jobs to only load records without a Salesforce Id already assigned. That will allow you to safely re-run the job if it encounters errors that lead to partial success (like record locking, see below).

Why Not Upsert?

People used to loading small amounts of data will be tempted to use Salesforce’s upsert command. The benefit is that it allows you to use those legacy Id values directly instead of swapping for the newly generated Id. But as record volumes grow, upsert performance drops – I’ve had projects where I measured it at ⅓ the speed of insert, I heard of projects where it got far worse than that. The larger the dataset, the more important it is to use Insert.

Playing Nice with Salesforce

To make sure your data loads correctly, and efficiently, there are three more important details you still need to plan for: 

  1. Automations and Sharing Rules
  2. Object Load Order
  3. Record Load Order

Automations and Sharing Rules

Automations take time to run, even small amounts of time add-up when loading large amounts of data. To the degree possible, you want automations off. Some automations you want to replicate in your transformation process – particularly if it’s a simple field value or record creation. Some automations you want to defer and run later, like custom roll up values via DLRS, NPSP rollups, or similar approaches. And some automations you cannot disable at all.

Sharing calculations in Salesforce are really a special-purpose automation. Just not one you often think about unless you’re doing manual sharing. Like all automations in Salesforce, the more data you load, the larger the impact of these calculations. Salesforce allows you to defer these calculations and run them in the future. The more complex your security setup, the more impact this will have (open security models can generally ignore this consideration).

The person doing the data loading needs to work with the folks that implemented those automations to map out which can be disabled, which can be deferred, and which must to be tolerated.

Object Load Order

In Salesforce, object load order is critical. You cannot disable or defer assignment of required references. So you need to understand the object hierarchy and relationships.

Generally you start with objects that have no dependencies: e.g. Account, Campaign, Product, Lead.

Then proceed to objects that have relationships to those: e.g. Contact

Then to objects that can have relationships to objects from that previous layer: e.g. Opportunity, Account Contact Relation, Campaign Member

When possible, test running two objects in parallel. What exact combination is most efficient will vary by org details and data volumes. My experience is that you will be able to run objects in 4-5 groups usually with two or three objects loading in parallel.

Ideally we’d just load records and not have to go back and update, but if there are circular references, or record hierarchies you’ll need to update records after insert. Plan that second pass into your sequence.

Users

Salesforce Users are a special case. If you have a security model where record ownership is important, you need to load Users first.  If you have an open security model, I recommend loading Users last – and the smallest number of Users possible.  Remember, Salesforce bans User deletion, so you must be as careful as possible about loading them.  I never like to load Experience Cloud Users if I can avoid it – 1,000’s of accounts that will never be used but cannot be deleted is sub-optimal.

Record Load Order and Record Locks

Salesforce has aggressive record locking to deal with concurrent edits and updates across relationships. Great for day-to-day operation; frustrating when you’re loading data.

The first place people often encounter this is when they go to load Opportunities. Opportunity bulk load can run into massive problems with Account records being locked because another Opportunity is being loading for the same Account in a parallel process. If you sort the records by the locking parent record you can often reduce, if not eliminate, your record locking issues.

Use Serial Mode only as a last resort. Serial mode is ⅕ the speed of Parallel mode most of the time. There are situations that call for it. But it should never be your default go-to solution. Try everything else first before resorting to serial mode. Since you have tracking of which records were loaded or failed, if you design your load job carefully you can just re-run to resolve small numbers of record locks.

Extra Sorting Trick:

It turns out, in many cases the way data gets entered over time will gather it in useful patterns. So sorting data by a date field can radically reduce record lock contention. If you cannot figure out what field to sort by (often because sorting by field 1 causes locking issues on another object) try sorting by a date field and see if that helps.

Warning: depending on your data patterns, it can make the problem vastly worse too.

Mock Runs

A mock run is a test load into a sandbox that should involve you going through all the steps to load the data – starting with extracting it from the source system.

I personally recommend at least two full test mocks of your process.

If you’re working on a tight budget that may not be feasible (migrations are the first place project leaders trim budgets, and the first place users complain about errors), but that doesn’t mean multiple tests aren’t valuable.

The first test will go poorly, but you’ll learn a lot. The second test will, hopefully, go far better, but you will still learn a great deal. 

In your testing you should expect to find places where your mappings are wrong, your transformations are incorrect, your testing is inadequate, your load order doesn’t work, you have source data patterns not accounted for, and more. Make time for good testing, you’ll thank yourself later.

Final Considerations

Large volume data loading in Salesforce is a deep topic. For all this is a long article, I’ve left out a lot of details. I designed this pattern to support high speed loads, rigorous testing, and error recovery. But within each of step of this pattern I could write articles this long or longer. You should continue to research the topic and adapt your implementation to your project.

A few sample topics you might consider:

You may even need to do something I’ve never encountered before.

But in any large volume Salesforce data load, the general pattern outlined here will serve you well.