Welcome to the presentation turbulent times c

hallenges in CFD my

name is flora mint IAM chief technologist at

the fluids

business unit and I want to talk a little bit

about

the difficulties we have in CFD computations

I thought I

started office comparison of two simple or se

emingly simple cases

one in structural mechanics but we have this

beam under

bending moment or one of the simpler problems

you can

have and as you expect for simple problem in

structural

mechanics you get a simple solution in this c

ase you

can get an analytical solution or you can sol

ve it

with a few finite elements on a Very small co

mputer

if you look at a fairly simple CFD problem we

have this square cylinder in across flew it a

lso looks

simple but in reality it's not cannot be done

analytically

and if you look into the details we actually

find

that at the end of the day what we have

is a high performance compute computing probl

em and I'll show

you why that is in this movie from this group

here what you see here that this A simple pro

blem

has an enormously complex solution we have se

parations from the

front part of the cylinder at the edges here

we

have mortice is rolling up there then you hav

e this

reattachment areas you have these large volta

ges forming in the

back of the cylinder and they have this chat

like

structures emanating here and all of that is

is far

from trivial obviously an what makes things e

ven more difficult

is that we have a strong dependency on the Re

ynolds

number of this flow The Reynolds number of th

e flow

is defined by the freestream velocity by the

lengths edge

lengths of the cylinder and by the molecular

viscosity of

the fluid and technical flows typically have

water or air

have low viscosity so we have relatively high

Reynolds numbers

and in this case it's a moderate Reynolds num

ber about

22,

000 and what we see here is a direct numerica

l

simulation of the navier Stokes equation so w

e resolve or

these people resolved all Time and length sca

les in that

flow and that required about 300 million cell

s and about

600,

000 time steps to get that solution on fluent

that

would equate to about 20 days on 1000 cores s

o

that's still manageable clearly but it's very

expensive and of

course that's a very small problem size and i

n addition

it's not a realistic Reynolds number The main

issue with

direct numerical simulation is that it scales

very strongly with

the Reynolds number so it scales with the Rey

nolds number

to the third power and if you go to a

more realistic technical Reynolds number of l

et's say 2.

2 million

that would result in 55,

000 years on 1000 core computation

computer so that's that's how this scales and

of course

that's astronomically and cannot be done with

any reasonable Resource

Now let's look at the more real flow more tec

hnical

flow we have this aircraft here and of course

here

we have them completely different range of di

mensions the aircraft

has a dimension of the order of 100 meters an

d

the dimension of the flow near the wall that

we

have a very thin bound layer with mortises he

re in

this near ball area because of no slip condit

ion at

the wall we get very strong gradients here an

d that

causes a lot of these Invoices to appear this

layer

near the wall is about a few millimeter to to

a few centimeters and inside that layer we ha

ve these

small vortices which near the wall of the ord

er of

10 to the minus 5 meters or even smaller and

appropriate decoding timescales so we have a

resolution problem which

is would require 10 to the 15th tend to the

18th cells to solve their problem and of cour

se that's

not realistic on any available computing powe

r if you Could

connect a lot of computing centers we would n

ot be

able to do that but on the other hand this

is all important because these small ages the

y determined the

aerodynamics of the aircraft so if they would

stop working

the aircraft would crash if he would have lem

on apparently

there it would just stall out into the aircra

ft would

not fly so we have to account for the effect

of these edges on the aerodynamics How could

we do

that obviously have to go to modeling if you

want

to have an engineering solution now here you

have the

navier Stokes equations here and this is the

momentum equation

so it's the conservation of momentum and if y

ou look

at the box we have a change of momentum we

have flow into and out of that box of momentu

m

we have a pressure force and we have viscous

forces

there and the main problem in navier Stokes i

s the

nonlinearity of the convective terms so the m

omentum is proportional

to velocity And it's transported by philosoph

y is that we

have a quadratic nonlinearity which causes al

l that chaotic nonlinear

motion there Now what we do is we typically w

ant

to know the mean values of our quantities so

we

split up the velocity fluctuation in a time m

ean value

and in the fluctuating velocity and then we t

ime averaged

equation and if we do that with this one term

we end up with the product of the mean values

and the product of the fluctuating components

and the product

of those over ball averaged is not cereal for

example

you squared is not zero obviously and the oth

ers are

also not 0 So we'll be introduced that into t

he

navier Stokes equations and we get this addit

ional term which

is called the Reynolds stress tensor and that

is basically

an interface of how we can feedback informati

on on turbulence

into the time averaged equations there and th

at's what's called

turbulence modeling and the whole process of

averaging and modeling

is called Reynolds averaged navier Stokes equ

ations or rans modeling

and of course we then need to bridge many man

y

orders of magnitude in computing power We sai

d technology and

it's clear that that can lead to significant

errors so

we cannot replace a complex problem by a simp

le one

and get the same accuracy as solving on a sup

ercomputer

So the way that looks like then is we have

here this additional term in the navier Stoke

s equations there

and we have to find some sort of a mathematic

al

formulation for that what we do is we make ve

ry

off the assumption that is run stress tensor

can be

modeled in analogy to the molecular stress te

nsor and we

have a viscosity times the velocity gradient

but of course

that viscosity here is not molecular but the

turbulent viscosity

and it comes from the edges So the the dimens

ion

of this viscosity is a length squared divided

by a

time scale and these lengths is the size of t

he

large edges so the structure of the large ide

as and

the time is the turnover time of the lodge it

is cause the large edits are responsible for

the mixing

process is and that's what we're interested n

ow we need

now to equations to describe this scales and

here I

have written a generic equation for the lengt

h scale And

the generic equation for the time scale so we

have

stand at the end of the day two transport equ

ations

mostly steady state equations which we can so

lve in addition

to the momentum equation so at very low cost

and

what we solve is not this complex flow here b

ut

we solved in this very smooth distributions o

f length scale

in time scale and compute the viscosity and t

hen we

feed that back into the navier Stokes equatio

ns and go

on until we have a steady state so use essent

ially

So a very elegant way of doing that but of

course the the problem is with these models h

ow accurate

is such a simple formulation that we have a f

ew

coefficients obviously which we can then use

to tune the

model but it's not always reliable and here i

s a

drastic example of a fairly simple slow so we

have

diffusers or just two pipes being merged by t

his diffuser

between those large diameter and small diamet

er and typically the

task of the engineers to avoid the separation

in that

diffuser because they Generates losses or noi

se or other problems

and then if

You run that flew visa one turbulence model y

ou get

that solution and if you run it is another tu

rbulence

model you get that solution in this case that

one

happens to be the correct one but how do you

know one which one is is correct so that has

a lot of implications this uncertainty there

on how CFD

is done and also on the level of knowledge a

user still has to have they have to be able

to judge which model is the most proper model

for

their cases And they also have to validate th

e code

and the method against cases which are simila

r to their

application that they want to apply that mode

l too so

if they have a simple diffusion flow they can

decide

OK that model is better than the other and th

en

they can apply it to their real flow

Now of course we be then have the problem tha

t

it's kind of random it takes that model and i

t

works so it doesn't and one of the areas we

have worked on in recent years is make these

modeling

approach is more flexible and what that means

is we

take it to equation model here it's a K Omega

type model so we have a K equation Omega equa

tion

is a turbulent frequency

And we introduced functions like these three

functions there and

dysfunctions are then designed to have free p

ara meters and

this free para meters can then be tuned by th

e

user in a very well defined way and they know

exactly what happens if they changed that coe

fficient and we

have documentation of how to do that and they

can

then tune that model to the specific applicat

ion they're interested

in

What's also important is that this coefficien

ts are all field

variables so they can not only be changed glo

bally but

they can be changed locali so for example if

you

think about the Formula One car you might hav

e a

different set of coefficients on the front wi

ng the aerodynamic

body there that you might have behind the fro

nt wheel

they have a lot of mixing and you might want

to have a much higher Ed viscosity there then

you

would want to have on the on the front wing

section so it's a social model And of course

at

the end of the day he can go even further

that we have these coefficients locali so in

each cell

we have different value which means we have a

field

of coefficients then which I'll talk a bit ab

out later

now what that means in the flexibility of suc

h a

model I show here for this airfoil at angle o

f

attack So what we do here we change the angle

of attack of the airfoil and we look at the

lift coefficient and we see the experiment li

ft goes up

to about 12 degrees and then it goes down

And if you take the orange curve is starting

point

of the gecko model which should be changed to

see

sepco efficient which changes the separation

characteristic of the model

we change it from 1.0 to 2.5 and we can

see we can assume the model to the experiment

al data

one has to say that from this point on the

experiments becomes 3D and the simulation is

2D so we

can't really match all the details here it's

more matching

that Max point here and if you look at a

certain angle of attack At the velocity profi

les near the

trailing edge we can see that the model is ab

le

to cover the To envelope the experimental dat

a with some

more conservative solution office and overly

aggressive solution there and

the truth is somewhere in between somewhere b

etween 1.

75 and

two is probably the optimal solution for that

angle of

attack here so that gives the users a much wi

der

ability to tune and calibrate the model than

was the

case with previous models and that's widely u

sed now and

people adopt that here's another example wher

e we have a

triangular cylinder in crossflow It's a very

extreme example actually

because as we have seen with this special in

the

first slide or second slide there is also ver

tex shading

here which is not really turbulent

But if you want to have a steady state soluti

on

and we use a default we get a very large

separation soon and if you look along this li

ne here

centerline we see the experiment has a very s

hort separation

so that philosophy is negative and the defaul

t would be

very large separation so we have to tune the

mixing

coefficient here and we can then tune it to a

gree

much better we don't exactly get back flow bu

t we

get the recovery much better so if you think

about

this Formula One car And you want to have a

proper flew towards the back wheel coming fro

m the wake

of the front wheel you would be much better o

ff

with this purple a curve then he would be wit

h

the red curve to get the aerodynamics of the

of

the back wheel so that's another example of h

ow the

model can be calibrated When doing that you h

ave to

do that by hand but ideally you would like in

the future to have a computing mechanism or s

trategy how

you could optimize these coefficients and cle

arly machine learning comes

to mind

And how would that work if you look again at

the lift coefficient as a function of one sin

gle parimeter

DC step coefficient in the turbulence model a

nd remember she

said is a field so in each cell we can

give it a different value we can define a err

or

and the error is the actual lift computed fro

m CFD

minus the lift coming from the experiment so

it's a

global value integrated over the airfoil

And now we want to minimize the error so we

have to know how does the error change if I

change the step in a certain cell and then I

have to find the optimal distribution of CSF

values to

get the error reduced and in order to do that

we use the chain rule here and then you need

that quantity here so that's how this CL chan

ge this

C step

And that's a very complex entity becausw it's

hard to

compute the impact of a single cell change to

the

overall integral value of the left difficult

efficient and that's

what an adjoint solver can do for us and we

have implemented influence and adjoint solver

of the entire gecko

model so we can compute that sensitivity and

then we

can update the field of C set based on that

sensitivity and we can get an optimal distrib

ution of the

sycip coefficient And that's what that looks

like here this

is the correction relative to default value s

o zero would

be default and we see it increases this excep

t coefficient

so we have we get more separation than what w

e

get is the default model but we would expect

pretty

much there

And then of course that gives us optimal dist

ributions and

we can do that for different angles of attack

and

for different airfoils so we get a lot of dat

a

but it's not really useful because you always

need to

have the experimental data to get that distri

bution but then

we can try whether we can correlate that so w

e

have to see set fields and we have non dimens

ional

quantities in each cell which we compute anyw

ay so here

strain rate divided by Omega or this non dime

nsional K

square OK romika bowl distance from beauty of

imu and

we can try whether there is a correlation so

when

we know that these ones here can we compute t

hat

one and that can be done by using your own

networks so the way that looks like we take a

neural network you take the training data whi

ch we know

this app distribution which we have optimized

by the adjoint

solver and we define the input quantities whi

ch we want

to use for the correlation so we put the inpu

t

quantities into the neural network and we get

to see

Step 4 cell Which is wrong but then we can

back substitute waits until the weights are b

asically balanced so

that given the input variables we get a good

approximation

of the known see step distribution and once w

e have

done that for number of training data we know

the

weights of the network and we can use the net

work

then as a predictive tool so we just feed in

this known quantities here and we get out acc

ept for

that computational cell And you can see we ca

n represent

the adjoint solution to the left quite well b

ased in

your own network to the right so we have obvi

ously

selected sensible input para meters and once

we have done

that we can then go and compute new cases thi

s

neural network and you can see you starting f

rom default

here we can get significant improvement of da

ta again this

is a 2D simulation against 3D data so we don'

t

get this full 3D fall off here but we get

a much better agreement of Blue curve against

the data

the other way to improve accuracy is by going

doing

it the hard way by resolving parts of this tu

rbulent

structures there and that technique is called

lottery simulation and

the idea is you only resolved largest 80s but

you

don't resolve the very small ladies because t

hey don't contribute

to the mixing and that of course requires a t

ime

integration of 3D integration and the main pr

oblem is there

that this is very expensive All because near

the ball

the large eddies become small because they ca

nnot be larger

than the wall distance for geometric reasons

so that works

fine for pre shear flows but it is pretty dif

ficult

to do that for all bounded flows now here's a

n

example of some of our colleagues at Boeing a

nd in

Russia which we work with have done this shoc

k boundary

layer flow so they have a marker .

875 flow goes

supersonic has a shockwave separates the flow

and has a

strong interaction so you can see here Initia

lly on picture

where the separation goes off and they have r

un half

a billion cell Ramesh then 1.

7 billion and at the

end of the day or close to 9 billion cell

problem to get that flow in here you can see

the movie The outer flow is fairly benign it

just

travels along but there's a lot of commotion

very close

to the wall and that determines the separatio

n characteristics of

that flow and what they found was with their

half

a billion and 1.

7 billion cell problems mesh one and

two They got pretty much the same solution an

d it

was pretty bad so the shark is on the wrong

location you look at pressure distribution he

re and the plateau

which signifies separation is also not there

and they started

to think the experiment there's something wro

ng there but then

they did the further Metra feynman tube out n

ine billion

cells and then they got pretty close to the d

ata

and you can see the separation size here chan

ges quite

drastically when they go from that mesh to th

at message

One of the reasons for their problems here is

that

in the accelerating region on the front part

of the

bump you have a thick boundary layer Cam in b

uttock

form a new boundary layer and it's much harde

r to

resolve this new boundary layer then to resol

ve the old

one so the initial estimates on the grid were

based

on that boundary layer thickness but in reali

ty you have

to resolve that one and that's where the cost

of

this simulation exploded literally in terms o

f mesh cells and

time steps

Now an alternative or in between engineering

approaches to Mitch

mix and match so the user rants turbulence mo

del and

we use an alias model and we define automatic

blending

functions to switch between them and then we

can have

models which in the boundary layer which is v

ery expensive

we can have rants and in the fresh air flows

we can resolve these edges and we can have an

alias simulation in these areas and that's a

pretty good

compromise And you can see here this semi rea

listic arbiza

your anger so side wind conditions and we can

match

that flow topology here quite reasonably comp

ared to the wall

streamlines if you look at these structures h

ere for example

also there's some structures there and so tha

t's somewhere in

cost significantly lower than the full large

Eddy simulation and

of course much higher than the then ran simul

ation this

is simulations here from Peter eckman

Now

CFT bill as you have now found out always be

driven by computing power so the more computi

ng power the

more ideas we can resolve and that will lead

us

at the end of the data very enormous mesh siz

es

the 10 billion you have seen for simple probl

em and

there is no limit to that so we can have

100 billion or trillions of cells in the futu

re

And in order to do that we need some organizi

ng

principles and we work on Oct reason doctors

are very

nice concept but an octree does it defines sp

ace with

a 1 dimensional curve so these six a curve to

uches

every single cell we can make here we can ref

ine

the cell and six axis more and you have a

very efficient way of storing space data so t

o speak

in a 1D integer Great and miss that octree on

e

can do a lot of things for example one could

do meshing like generating these meshes here

and you can

do that in a very scalable fast way on parall

el

computers with a relatively low memory count

or we can

use this also for other tasks in HPC for exam

ple

parallel partitioning measure rotation load b

alancing zoom in zoom out

capabilities during post processing data redu

ction for transfer and other

things And you can speed up your solver or if

you know you have a condition message you can

speed

it up to simplify the solver and you can also

go to alternative solvers like lettuce bolts

month running on

these types of measures days also in the curr

ent series

presentation by Dominic salts which gives a l

ot of details

on the octree so if you have time you can

also look at that

Now here is an impression of meshes generated

again for

this car this is a top down mesh so it

does something featuring but the mesh on the

wall for

Elias is so fine for all function areas in th

is

case that the feature is not really playing a

ny significant

role and then you can do different things her

e for

example you made one protection layer here th

e other wall

so we have body fitted meshes and here you ca

n

see the scaling of the matching process so th

e mesh

100 million cells was generated On 256 cores

in the

30 seconds and that is not the limit so we

can scale that up to many many thousands and

10

thousands of cores depending on the mesh size

obviously and

have a very good way in passing to the high

performance computing using that technology h

ere you can see the

solutions of wall function alias on the cours

e very coarse

mesh 7 million 35 million 100 million we had

a

bit of a problem here numerically we have res

olved that

Issue there and you can see the more mesh you

invest the most structures you get and then t

he solution

develops into this vague region here across t

he backs land

and if you compare them to the experimental d

ata here

on the roof is actually pretty good even the

7

million mesh solution is not terrible and the

n as you

go across this land of course the 7 million f

ades

a little bit but then gradually it increases

and improves

as we refine the mesh that you have to really

be careful what you do here And we also have

included this hybrid runs alias method that I

talked about

before which is significantly cheaper and act

ually in this case

is the best solution at least the black curve

there

so we have a lot of different options there i

n

how we do this care resolving simulations and

I think

we in the future will go more and more into

this large ID and all function areas simulati

ons that I've

shown

Now that brings us to the path forward I thin

k

in rents clearly the path forward will be mac

hine learning

currently the last 50 years we have tuned the

models

by hand and we went probably as far as we

reasonably could and now the question is can

be improved

that by having machines that work for us into

the

calibration there and we can also use trainin

g data sets

not only experimental but we can also generat

e training data

Using large Eddy simulation and or DNS even a

nd use

these data to train the neural networks for s

pecific generic

flows as I said I expect horses for courses t

raining

so different types training for different typ

es of applications there

even though it would be nice to have a generi

c

model but I don't expect that to really happe

n and

the other path forward is for many years to c

ome

with scale resolving simulations computationa

l speed is essential so we

need all the combinations of fast solver Nume

rics high scalability

in parallel machines for all elements of the

simulation modern

computing architectures like GPUs and who kno

ws maybe eventually quantum

computing so that's all I had to say here tha

nk

you for your attention