An example for integration by parts.

TL;DR:

We break up the steps involved in computing the following integral: $$ \int e^x \sin x \, dx $$

Target Audience:

High school students, olympiad enthusiasts.

Why:

Integration (as also differentiation) is one of the essential tools in a machine learning practitioner's toolbox. One of the primary techniques utilized in integration is integration by parts, and to high schoolers applications of integration by parts often appear somewhat magical. The material for this article arose while discussing the problem with a friend in high school. The desire in this article is to

I plan to relegate some of the discussion in a followup article, just so that the current article is the right byte-sized.

How:

This is how we will proceed wrt this topic.

The techniques at hand.

We will assume here that the reader is aware of some of the more effective techniques in integration: integration by parts and integration by substitution.

Out of the two techniques mentioned, integration by substitution is perhaps the simpler one. It is a kind of form-fitting where it leaps to the eye that a smart substitution can solve the integral for us. For instance, looking at $$ \int \frac{\ln x}{x}\, dx $$ we might realize that $\frac{d}{dx} (\ln x) = \frac{1}{x}$, and so substituting $t = \ln x$ will solve the problem for us (I leave it to the reader to complete solving this indefinite integral).

Integration by parts, in comparison, is slightly more sophisticated. This is how integration by parts generally reads: given functions $u$ and $v$, $$ \int u \, dv = uv - \int v \, du $$ While this is the form that high schoolers may be usually familiar with is, it also helps to remember the more symmetric form of the equality above: $$ uv = \int u \, dv + \int v \, du $$ The basic idea behind integration by parts is this: it may happen that $\int u \, dv$ strikes no bells, and might be harder to integrate directly (or by substitution), we might be able to solve the easier $\int v \, du$.

The problem:

The problem is to compute the (indefinite) integral $I = \int e^x \sin x \, dx$. Considering the form of the integrand, we are unable to directly apply substitution to the problem. Given this, it might make sense to try the other technique in our repertoire - integration by parts! Ok, so let's start.

Solution 1 - Start off:

Once we have decided to apply integration by parts, we realise that we have a couple of options here. Either $dv = e^x \, dx$ or $dv = \sin x\, dx$. In the former case, $v = e^x$, while in the latter $v = \cos x$. Let us pursue both of these possibilities as independent threads:

  1. $dv = e^x \, dx$, $u = \sin x$ gives $v = e^x$ and $du = \cos x\, dx$. So plugging this into the equation for integration by parts, we get: $$ I = \int u\, dv = e^x \sin x - \int e^x \cos x\, dx $$
  2. $dv = \sin x\, dx$, $u = e^x$ gives $v = -\cos x$ and $du = e^x\, dx$. Plugging these values into the equation for integration by parts again, we get: $$ I = \int u\, dv = - e^x \cos x - \int e^x (- \cos x)\, dx = -e^x \cos x + \int e^x \cos x \, dx $$

And... finish!

Ok, so now we have two expressions for $I$ and the very interesting thing is that, in one of the expressions we have $\int e^x \cos x dx$ while in the other we have the same integral with an opposite sign! So if we add the two expressions for $I$, the pesky integral cancels away, and we get (and remembering to place the arbitrary constant $c'$): $$ 2I = e^x (\sin x - \cos x) + c' $$ which brings us to the solution: $$ I = \frac{e^x (\sin x - \cos x)}{2} + c. $$ Done. Finito. (No magic about $c'$ and $c$. We just wanted to reserve c for the arbitrary constant in the last line; if you want, $c' = 2c$.)

Ok.. that was a bit too quick!

Yes, it was! We applied integration by parts twice, considering different parts of the integrand as $u$ or $v$ et voilà, we added the result and got the answer!

Let's slow this process down. Suppose it hadn't struck us that we could apply integration by parts twice and change the roles between $u$ and $v$, what could we do then?

So, suppose we had applied integration by parts only once, and got $$ I = \int u\, dv = e^x \sin x - \int e^x \cos x\, dx $$ It looks that we are stumped here, since, if we applied another integration by parts to solve/simplify $\int e^x \cos x\, dx$, we would get back $\int e^x \sin x\, dx$ and no progress would be made.

Well at least let's compile the new information we do have. Let us pull the integral $\int e^x \cos x dx$ on the RHS over to the LHS and collect terms to get (along with the obligatory arbitrary constant $c$): $$ I_1 = \int e^x (\sin x + \cos x)\, dx = e^x \sin x + c $$

The bigger picture?

From the specific example in $I_1$, we want to tease out a more methodical way of approaching the problem.
Generalize!
One constant effort in Mathematics is to generalize any situation that we come across. In the integration by parts above, we see that $e^x$ factorizes nicely precisely because $\frac{d}{dx}(e^x) = e^x$. We see that the situation in the integral $I_1$ above generalizes to an arbitrary function $f(x)$ (instead of $f(x) = \sin x$) as follows: $$ I_2 = \int e^x [f(x) + f'(x)]\, dx = e^x f(x) + c $$ This is an important identity that is useful to remember, and comes handy for many problems in indefinite integrals.
Aside: Note that this identity aligns with the (second) pattern for integration by parts, i.e. $$ uv = \int u \, dv + \int v \, du $$
Symmetry
So the one identity we have for $I_1$ is by instantiating $f(x) = \sin x$ in the identity for $I_2$. Why stop here, and why not instantiate $f(x) = \cos x$ too? One may think of $\sin x$ and $\cos x$ as sibling functions, so if we are applying some result (like for $I_2$ above) to $\sin x$, why not apply the same to $\cos x$ too? This yields $$ I_3 = \int e^x(\cos x - \sin x)\, dx = e^x \cos x + c $$
And finish.
Once again, we have $I_1$ and $I_3$ - look again at the LHS of either equation. If we subtract $I_3$ from $I_1$, we get the integral (well $2\times$) that we require, i.e. $\int e^x \sin x\, dx$. Once again we are done with the problem!

Yet another solution - the real Solution 2!

Note that in both of the solutions above, we had to realise that we can apply integration by parts, and then we also had to apply it twice. So in a sense, both of the solutions above are the same, just that the second one is some sort of an unrolling of the first nifty-looking solution.

Given this, let us call the solutions presented above as Solution 1, and the current one as Solution 2.

Suppose it did not strike us to make that second application of integration by parts - what then?

Okay, let's take stock. It does need to strike that we will have to apply integration by parts - once. But after that one application, we are stymied; we have arrived at this equality: $$ I_1 = \int e^x (\sin x + \cos x)\, dx = e^x \sin x + c $$ And (suppose) we do not realise that we have to apply integration by parts a second time (in a manner described above).

When we are stymied, its always good to look back and fiddle around with what we already have. In this case, let us try to present what we have more succinctly. To this end, let us try to understand the expression $(\sin x + \cos x)$. What we mean by "understanding" here is that, we would like to see this expression expressed as a single sine or cosine (instead of both of them appearing). In other words, we would like to "simplify" the expression containing both a sine and a cosine into one that just has a sine (or a cosine).

At this point, we need to think back on the trigonometric identities that we know. Where does this happen? Oh yeah, precisely in the expansion of $\sin (A + B)$ (or $\cos (A + B)$). Recall the identity: $$ \sin (A + B) = \sin A \cos B + \sin B \cos A $$ This is clearly an example where lots of sines and cosines on the RHS are summarized into a single sine on the LHS.

With this background, we may now manipulate our expression as follows: $$ (\sin x + \cos x) = \sqrt{2}\, \left(\frac{1}{\sqrt{2}}\sin x + \frac{1}{\sqrt{2}}\cos x\right) = \sqrt{2}\, \sin\left(x + \frac{\pi}{4}\right) $$ So we may rewrite the identity for $I_1$ as: $$ I_1 = \int e^x (\sin x + \cos x)\, dx = \sqrt{2}\, \int e^x \, \sin\left(x + \frac{\pi}{4}\right) \, dx = e^x \sin x + c $$

Let us extract out the essence of this last identity as follows: $$ I_4 = \int e^x \, \sin\left(x + \frac{\pi}{4}\right) \, dx = \frac{1}{\sqrt{2}}\, e^x \sin x + c $$ (where we are being sloppy about the arbitrary constant $c$, the $c$ for $I_4$ is a scaling factor away from the $c$ for $I_1$.

Personally I find this identity for $I_4$ rather beautiful. What this means is that (modulo the scaling factors $\frac{1}{\sqrt{2}}$), integrating the LHS leads to a phase change by $\frac{\pi}{4}$.

And now we can get one more solution to our original problem! Stare hard at the last identity for $I_4$. The integrand there is quite close to the integrand we want: $e^x \sin x$. So we realise that we do not need integration by parts any more, but instead integration by substitution! Let us proceed to substitute $t = (x + \frac{\pi}{4})$. In terms of $t$, the identity above reads: $$ I_4 = \int e^{\, t -\frac{\pi}{4}} \sin t \, dt = \frac{1}{\sqrt{2}} e^{\, t -\frac{\pi}{4}} \sin \left( t - \frac{\pi}{4} \right) + c $$ and now cancel the $e^{-\frac{\pi}{4}}$ terms on either side, and we are done!

A third solution?

Given all of the above, there is a third solution that involves (a) neither integration by parts, nor (b) integration by substitution. This post is already getting way too long, and I will relegate the third solution to another post. The reader is invited to taste the teaser in the takeaways below.

Exercises for the reader

Here are some other problems that may either be solved by following the techniques above or may be directly reduced to the problem above. I leave it to the reader to figure these out and outline the solution in the comments section below.
  1. Solve the indefinite integral $\int e^x (\sin x + 2\cos x) \, dx$.
  2. Solve the indefinite integral $\int e^{2x}\sin x \, dx$.
  3. How about $\int e^x \sin^2 x\, dx$? $\int e^x \sin^3 x \,dx$? In general, we can solve for $\int e^x \, q(\sin x) \, dx$ where $q(\cdot)$ is an arbitrary polynomial. To stretch this, we can in fact solve for $\int p(e^x) \, q(\sin x) \, dx$ where $p$ and $q$ are both arbitrary polynomials!
Hopefully the reader gets the idea as to the number of other indefinite integrals that now fall within reach.

Takeaways



Created 13 July 2017.