Labelled Markov Processes - cs.cornell.edu
Transcript of Labelled Markov Processes - cs.cornell.edu
![Page 1: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/1.jpg)
Introduction
Labelled Markov ProcessesA tutorial overview
Prakash Panangaden
School of Computer ScienceMcGill University
12th June 2014, Cornell University
Panangaden (McGill) Labelled Markov Processes MFPS XXX 1 / 41
![Page 2: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/2.jpg)
Introduction
Collaborators
Josée Desharnais
Radha Jagadeesan and Vineet GuptaAbbas EdalatPhilippe Chaput, Vincent Danos, Gordon PlotkinFrano̧is LavioletteNorm Ferns, Doina Precup, Gheorghe ComaniciDexter Kozen, Kim Larsen, Radu Mardare
Panangaden (McGill) Labelled Markov Processes MFPS XXX 2 / 41
![Page 3: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/3.jpg)
Introduction
Collaborators
Josée DesharnaisRadha Jagadeesan and Vineet Gupta
Abbas EdalatPhilippe Chaput, Vincent Danos, Gordon PlotkinFrano̧is LavioletteNorm Ferns, Doina Precup, Gheorghe ComaniciDexter Kozen, Kim Larsen, Radu Mardare
Panangaden (McGill) Labelled Markov Processes MFPS XXX 2 / 41
![Page 4: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/4.jpg)
Introduction
Collaborators
Josée DesharnaisRadha Jagadeesan and Vineet GuptaAbbas Edalat
Philippe Chaput, Vincent Danos, Gordon PlotkinFrano̧is LavioletteNorm Ferns, Doina Precup, Gheorghe ComaniciDexter Kozen, Kim Larsen, Radu Mardare
Panangaden (McGill) Labelled Markov Processes MFPS XXX 2 / 41
![Page 5: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/5.jpg)
Introduction
Collaborators
Josée DesharnaisRadha Jagadeesan and Vineet GuptaAbbas EdalatPhilippe Chaput, Vincent Danos, Gordon Plotkin
Frano̧is LavioletteNorm Ferns, Doina Precup, Gheorghe ComaniciDexter Kozen, Kim Larsen, Radu Mardare
Panangaden (McGill) Labelled Markov Processes MFPS XXX 2 / 41
![Page 6: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/6.jpg)
Introduction
Collaborators
Josée DesharnaisRadha Jagadeesan and Vineet GuptaAbbas EdalatPhilippe Chaput, Vincent Danos, Gordon PlotkinFrano̧is Laviolette
Norm Ferns, Doina Precup, Gheorghe ComaniciDexter Kozen, Kim Larsen, Radu Mardare
Panangaden (McGill) Labelled Markov Processes MFPS XXX 2 / 41
![Page 7: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/7.jpg)
Introduction
Collaborators
Josée DesharnaisRadha Jagadeesan and Vineet GuptaAbbas EdalatPhilippe Chaput, Vincent Danos, Gordon PlotkinFrano̧is LavioletteNorm Ferns, Doina Precup, Gheorghe Comanici
Dexter Kozen, Kim Larsen, Radu Mardare
Panangaden (McGill) Labelled Markov Processes MFPS XXX 2 / 41
![Page 8: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/8.jpg)
Introduction
Collaborators
Josée DesharnaisRadha Jagadeesan and Vineet GuptaAbbas EdalatPhilippe Chaput, Vincent Danos, Gordon PlotkinFrano̧is LavioletteNorm Ferns, Doina Precup, Gheorghe ComaniciDexter Kozen, Kim Larsen, Radu Mardare
Panangaden (McGill) Labelled Markov Processes MFPS XXX 2 / 41
![Page 9: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/9.jpg)
Introduction
Summary of Results
Probabilistic bisimulation can be defined for continuousstate-space systems. [LICS97]Logical characterization. [LICS98,Info and Comp 2002]Metrics. [CONCUR99, TCS2004, UAI 2004, UAI 2005, SIAM J.Comp. 2011, QEST 2012]Approximation of LMPs. [LICS00,Info and Comp 2003, QEST2005]Weak bisimulation. [LICS02,CONCUR02]Real time. [QEST 2004, JLAP 2003, LMCS 2006]Event bisimulation [CMCS 2004, Info and Comp 2006]Duality [LICS 2013, MFCS 2013, MFPS 2014]Approximation by averaging [CONCUR 2003, ICALP 2009, JACM2014]Logic and approximation [MFCS 2012]
Panangaden (McGill) Labelled Markov Processes MFPS XXX 3 / 41
![Page 10: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/10.jpg)
Discrete probabilistic transition systems
Definition
Just like a labelled transition system with probabilities associatedwith the transitions.
(S,L,∀a ∈ L Ta : S× S −→ [0, 1])
The model is reactive: All probabilistic data is internal - noprobabilities associated with environment behaviour.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 4 / 41
![Page 11: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/11.jpg)
Discrete probabilistic transition systems
Definition
Just like a labelled transition system with probabilities associatedwith the transitions.
(S,L, ∀a ∈ L Ta : S× S −→ [0, 1])
The model is reactive: All probabilistic data is internal - noprobabilities associated with environment behaviour.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 4 / 41
![Page 12: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/12.jpg)
Discrete probabilistic transition systems
Definition
Just like a labelled transition system with probabilities associatedwith the transitions.
(S,L, ∀a ∈ L Ta : S× S −→ [0, 1])
The model is reactive: All probabilistic data is internal - noprobabilities associated with environment behaviour.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 4 / 41
![Page 13: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/13.jpg)
Discrete probabilistic transition systems
Examples of PTSs
s0a[ 1
4 ]
��
a[ 34 ]
��s1 s2
a[1]��
s3
s0a[1]
��
b[1]
��s1
c[ 12 ]
��
c[ 12 ]
&&
s2
a[1]��
s4 s3A1 A2
Panangaden (McGill) Labelled Markov Processes MFPS XXX 5 / 41
![Page 14: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/14.jpg)
Discrete probabilistic transition systems
Bisimulation for PTS: Larsen and Skou
Consider
t0a[ 1
3 ]
��
a[ 23 ]
��t1 t2
b[1]��
t3
s0a[ 1
3 ]
��a[ 1
3 ]
��
a[ 13 ]
��s1 s2
b[1]��
s3
b[1]��s4
P1 P2
Should s0 and t0 be bisimilar?
Yes, but we need to add the probabilities.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 6 / 41
![Page 15: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/15.jpg)
Discrete probabilistic transition systems
Bisimulation for PTS: Larsen and Skou
Consider
t0a[ 1
3 ]
��
a[ 23 ]
��t1 t2
b[1]��
t3
s0a[ 1
3 ]
��a[ 1
3 ]
��
a[ 13 ]
��s1 s2
b[1]��
s3
b[1]��s4
P1 P2
Should s0 and t0 be bisimilar?Yes, but we need to add the probabilities.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 6 / 41
![Page 16: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/16.jpg)
Discrete probabilistic transition systems
The Official Definition
Let S = (S,L,Ta) be a PTS. An equivalence relation R on S is abisimulation if whenever sRs′, with s, s′ ∈ S, we have that for alla ∈ A and every R-equivalence class, A, Ta(s,A) = Ta(s′,A).The notation Ta(s,A) means “the probability of starting from s andjumping to a state in the set A.”Two states are bisimilar if there is some bisimulation relation Rrelating them.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 7 / 41
![Page 17: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/17.jpg)
Labelled Markov processes
What are labelled Markov processes?
Labelled Markov processes are probabilistic versions of labelledtransition systems. Labelled transition systems where the finalstate is governed by a probability distribution - no otherindeterminacy.
All probabilistic data is internal - no probabilities associated withenvironment behaviour.We observe the interactions - not the internal states.In general, the state space of a labelled Markov process maybe a continuum.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 8 / 41
![Page 18: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/18.jpg)
Labelled Markov processes
What are labelled Markov processes?
Labelled Markov processes are probabilistic versions of labelledtransition systems. Labelled transition systems where the finalstate is governed by a probability distribution - no otherindeterminacy.All probabilistic data is internal - no probabilities associated withenvironment behaviour.
We observe the interactions - not the internal states.In general, the state space of a labelled Markov process maybe a continuum.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 8 / 41
![Page 19: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/19.jpg)
Labelled Markov processes
What are labelled Markov processes?
Labelled Markov processes are probabilistic versions of labelledtransition systems. Labelled transition systems where the finalstate is governed by a probability distribution - no otherindeterminacy.All probabilistic data is internal - no probabilities associated withenvironment behaviour.We observe the interactions - not the internal states.
In general, the state space of a labelled Markov process maybe a continuum.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 8 / 41
![Page 20: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/20.jpg)
Labelled Markov processes
What are labelled Markov processes?
Labelled Markov processes are probabilistic versions of labelledtransition systems. Labelled transition systems where the finalstate is governed by a probability distribution - no otherindeterminacy.All probabilistic data is internal - no probabilities associated withenvironment behaviour.We observe the interactions - not the internal states.In general, the state space of a labelled Markov process maybe a continuum.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 8 / 41
![Page 21: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/21.jpg)
Labelled Markov processes
Motivation
Model and reason about systems with continuous state spaces orcontinuous time evolution or both.
hybrid control systems; e.g. flight management systems.telecommunication systems with spatial variation; e.g. cell phonesperformance modelling,continuous time systems,probabilistic process algebra with recursion.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 9 / 41
![Page 22: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/22.jpg)
Labelled Markov processes
An Example of a Continuous-State System
���������CCCCCCCCO
@@@I
a - turn left
b - turn right
c - straight
Panangaden (McGill) Labelled Markov Processes MFPS XXX 10 / 41
![Page 23: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/23.jpg)
Labelled Markov processes
Actions
a - turn left, b - turn right, c - keep on courseThe actions move the craft sideways with some probability distributionson how far it moves. The craft may “drift” even with c. The action a (b)must be disabled when the craft is too near the left (right) boundary.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 11 / 41
![Page 24: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/24.jpg)
Labelled Markov processes
Schematic of Example
L
a,c !! a,c-- M
a,b,c==b,c
mma,c
-- R
b,c}}
b,cmm
This picture is misleading: unless very special conditions hold theprocess cannot be compressed into an equivalent (?) finite-statemodel. In general, the transition probabilities should depend onthe position.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 12 / 41
![Page 25: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/25.jpg)
Labelled Markov processes
Stochastic Kernels
A stochastic kernel (Markov kernel) is a function h : S× Σ −→ [0, 1]with (a) h(s, ·) : Σ −→ [0, 1] a (sub)probability measure and (b)h(·,A) : X −→ [0, 1] a measurable function.
Though apparantly asymmetric, these are the stochasticanalogues of binary relationsand the uncountable generalization of a matrix.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 13 / 41
![Page 26: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/26.jpg)
Labelled Markov processes
Stochastic Kernels
A stochastic kernel (Markov kernel) is a function h : S× Σ −→ [0, 1]with (a) h(s, ·) : Σ −→ [0, 1] a (sub)probability measure and (b)h(·,A) : X −→ [0, 1] a measurable function.Though apparantly asymmetric, these are the stochasticanalogues of binary relations
and the uncountable generalization of a matrix.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 13 / 41
![Page 27: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/27.jpg)
Labelled Markov processes
Stochastic Kernels
A stochastic kernel (Markov kernel) is a function h : S× Σ −→ [0, 1]with (a) h(s, ·) : Σ −→ [0, 1] a (sub)probability measure and (b)h(·,A) : X −→ [0, 1] a measurable function.Though apparantly asymmetric, these are the stochasticanalogues of binary relationsand the uncountable generalization of a matrix.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 13 / 41
![Page 28: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/28.jpg)
Labelled Markov processes
Formal Definition of LMPs
An LMP is a tuple (S,Σ,L,∀α ∈ L.τα) where τα : S×Σ −→ [0, 1] is atransition probability function such that∀s : S.λA : Σ.τα(s,A) is a subprobability measureand∀A : Σ.λs : S.τα(s,A) is a measurable function.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 14 / 41
![Page 29: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/29.jpg)
Labelled Markov processes
Example of LMP
initial state
[0, 1)
a[U(1,2)4 ]
zz
a[x4
U(2, 3)]
$$
a[ x+U(0,1)4 ]
��
a[1−x4 ]
��(1, 2]
a
��
1a[U(2,3)
4 ]
//
a[U(1,2)4 ]
oo
a[1+U(0,1)4 ]
LL
(2, 3]
β
��s t
For x ∈ [0, 1), τa(x, [2.1, 2.4]) = x4 0.3
Panangaden (McGill) Labelled Markov Processes MFPS XXX 15 / 41
![Page 30: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/30.jpg)
Labelled Markov processes
Larsen-Skou Bisimulation
Let S = (S, i,Σ, τ) be a labelled Markov process. An equivalencerelation R on S is a bisimulation if whenever sRs′, with s, s′ ∈ S, wehave that for all a ∈ A and every R-closed measurable set A ∈ Σ,τa(s,A) = τa(s′,A).Two states are bisimilar if they are related by a bisimulationrelation.Can be extended to bisimulation between two different LMPs.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 16 / 41
![Page 31: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/31.jpg)
Labelled Markov processes
Larsen-Skou Bisimulation - Example
sα .3
��a .3
��
a .3
��4
α
��b .4
��
b .1
��
• •
• ?
b
NN ?b
oo
∼
tα .3
��
a .6
��4
a��
b .5
��
•
• ?
b��?
b
NN
Panangaden (McGill) Labelled Markov Processes MFPS XXX 17 / 41
![Page 32: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/32.jpg)
Labelled Markov processes
Logical Characterization
L ::== T|φ1 ∧ φ2|〈a〉qφ
We say s |= 〈a〉qφ iff
∃A ∈ Σ.(∀s′ ∈ A.s′ |= φ) ∧ (τa(s,A) > q).
Two systems are bisimilar iff they obey the same formulas of L.[DEP 1998 LICS, I and C 2002]
Panangaden (McGill) Labelled Markov Processes MFPS XXX 18 / 41
![Page 33: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/33.jpg)
Labelled Markov processes
Logical Characterization
L ::== T|φ1 ∧ φ2|〈a〉qφ
We say s |= 〈a〉qφ iff
∃A ∈ Σ.(∀s′ ∈ A.s′ |= φ) ∧ (τa(s,A) > q).
Two systems are bisimilar iff they obey the same formulas of L.[DEP 1998 LICS, I and C 2002]
Panangaden (McGill) Labelled Markov Processes MFPS XXX 18 / 41
![Page 34: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/34.jpg)
Labelled Markov processes
Event bisimulation
In measure theory one should focus on measurable sets ratherthan on points.
Vincent Danos proposed the idea of event bisimulation, which wasdeveloped by him and Desharnais, Laviolette and P.
Event BisimulationGiven a LMP (X,Σ, τa), an event-bisimulation is a sub-σ-algebra Λ ofΣ such that (X,Λ, τa) is still an LMP.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 19 / 41
![Page 35: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/35.jpg)
Labelled Markov processes
Event bisimulation
In measure theory one should focus on measurable sets ratherthan on points.Vincent Danos proposed the idea of event bisimulation, which wasdeveloped by him and Desharnais, Laviolette and P.
Event BisimulationGiven a LMP (X,Σ, τa), an event-bisimulation is a sub-σ-algebra Λ ofΣ such that (X,Λ, τa) is still an LMP.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 19 / 41
![Page 36: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/36.jpg)
Labelled Markov processes
Event bisimulation
In measure theory one should focus on measurable sets ratherthan on points.Vincent Danos proposed the idea of event bisimulation, which wasdeveloped by him and Desharnais, Laviolette and P.
Event BisimulationGiven a LMP (X,Σ, τa), an event-bisimulation is a sub-σ-algebra Λ ofΣ such that (X,Λ, τa) is still an LMP.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 19 / 41
![Page 37: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/37.jpg)
Metrics
Process Equivalence is Fundamental
Markov chains:LumpabilityLabelled Markov processes: BisimulationMarkov decision processes: BisimulationLabelled Concurrent Markov Chains with τ transitions: WeakBisimulation
Panangaden (McGill) Labelled Markov Processes MFPS XXX 20 / 41
![Page 38: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/38.jpg)
Metrics
But...
In the context of probability is exact equivalence reasonable?
We say “no”. A small change in the probability distributions mayresult in bisimilar processes no longer being bisimilar though theymay be very “close” in behaviour.Instead one should have a (pseudo)metric for probabilisticprocesses.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 21 / 41
![Page 39: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/39.jpg)
Metrics
But...
In the context of probability is exact equivalence reasonable?We say “no”. A small change in the probability distributions mayresult in bisimilar processes no longer being bisimilar though theymay be very “close” in behaviour.
Instead one should have a (pseudo)metric for probabilisticprocesses.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 21 / 41
![Page 40: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/40.jpg)
Metrics
But...
In the context of probability is exact equivalence reasonable?We say “no”. A small change in the probability distributions mayresult in bisimilar processes no longer being bisimilar though theymay be very “close” in behaviour.Instead one should have a (pseudo)metric for probabilisticprocesses.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 21 / 41
![Page 41: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/41.jpg)
Metrics
A metric-based approximate viewpoint
Move from equality between processes to distances betweenprocesses (Jou and Smolka 1990).
Formalize distance as a metric:
d(s, s) = 0, d(s, t) = d(t, s), d(s, u) ≤ d(s, t) + d(t, u).
Quantitative analogue of an equivalence relation.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 22 / 41
![Page 42: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/42.jpg)
Metrics
A metric-based approximate viewpoint
Move from equality between processes to distances betweenprocesses (Jou and Smolka 1990).Formalize distance as a metric:
d(s, s) = 0, d(s, t) = d(t, s), d(s, u) ≤ d(s, t) + d(t, u).
Quantitative analogue of an equivalence relation.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 22 / 41
![Page 43: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/43.jpg)
Metrics
Summary of results
Establishing closeness of states: Coinduction
Distinguishing states: Real-valued modal logicsEquational and logical views coincide: Metrics yield samedistances as real-valued modal logicsCompositional reasoning by Non-Expansivity.Process-combinators take nearby processes to nearby processes.
d(s1, t1) < ε1, d(s2, t2) < ε2
d(s1 || s2, t1 ||t2) < ε1 + ε2
Results work for Markov chains, Labelled Markov processes,Markov decision processes and Labelled Concurrent Markovchains with τ -transitions.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 23 / 41
![Page 44: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/44.jpg)
Metrics
Summary of results
Establishing closeness of states: CoinductionDistinguishing states: Real-valued modal logics
Equational and logical views coincide: Metrics yield samedistances as real-valued modal logicsCompositional reasoning by Non-Expansivity.Process-combinators take nearby processes to nearby processes.
d(s1, t1) < ε1, d(s2, t2) < ε2
d(s1 || s2, t1 ||t2) < ε1 + ε2
Results work for Markov chains, Labelled Markov processes,Markov decision processes and Labelled Concurrent Markovchains with τ -transitions.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 23 / 41
![Page 45: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/45.jpg)
Metrics
Summary of results
Establishing closeness of states: CoinductionDistinguishing states: Real-valued modal logicsEquational and logical views coincide: Metrics yield samedistances as real-valued modal logics
Compositional reasoning by Non-Expansivity.Process-combinators take nearby processes to nearby processes.
d(s1, t1) < ε1, d(s2, t2) < ε2
d(s1 || s2, t1 ||t2) < ε1 + ε2
Results work for Markov chains, Labelled Markov processes,Markov decision processes and Labelled Concurrent Markovchains with τ -transitions.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 23 / 41
![Page 46: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/46.jpg)
Metrics
Summary of results
Establishing closeness of states: CoinductionDistinguishing states: Real-valued modal logicsEquational and logical views coincide: Metrics yield samedistances as real-valued modal logicsCompositional reasoning by Non-Expansivity.Process-combinators take nearby processes to nearby processes.
d(s1, t1) < ε1, d(s2, t2) < ε2
d(s1 || s2, t1 ||t2) < ε1 + ε2
Results work for Markov chains, Labelled Markov processes,Markov decision processes and Labelled Concurrent Markovchains with τ -transitions.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 23 / 41
![Page 47: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/47.jpg)
Metrics
Summary of results
Establishing closeness of states: CoinductionDistinguishing states: Real-valued modal logicsEquational and logical views coincide: Metrics yield samedistances as real-valued modal logicsCompositional reasoning by Non-Expansivity.Process-combinators take nearby processes to nearby processes.
d(s1, t1) < ε1, d(s2, t2) < ε2
d(s1 || s2, t1 ||t2) < ε1 + ε2
Results work for Markov chains, Labelled Markov processes,Markov decision processes and Labelled Concurrent Markovchains with τ -transitions.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 23 / 41
![Page 48: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/48.jpg)
Metrics
Criteria on Metrics
Soundness:d(s, t) = 0⇔ s, t are bisimilar
Stability of distance under temporal evolution:“Nearby states stayclose forever.”Metrics should be computable (efficiently?).
Panangaden (McGill) Labelled Markov Processes MFPS XXX 24 / 41
![Page 49: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/49.jpg)
Metrics
Bisimulation Recalled
Let R be an equivalence relation. R is a bisimulation if: s R t if:
(s −→ P)⇒ [t −→ Q,P =R Q]
(t −→ Q)⇒ [s −→ P,P =R Q]
where P =R Q if(∀R− closed E) P(E) = Q(E)
Panangaden (McGill) Labelled Markov Processes MFPS XXX 25 / 41
![Page 50: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/50.jpg)
Metrics
A putative definition of a metric-bisimulation
m is a metric-bisimulation if: m(s, t) < ε⇒:
s −→ P⇒ t −→ Q, m(P,Q) < ε
t −→ Q⇒ s −→ P, m(P,Q) < ε
Problem: what is m(P,Q)? — Type mismatch!!Need a way to lift distances from states to a distances ondistributions of states.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 26 / 41
![Page 51: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/51.jpg)
Metrics
A putative definition of a metric-bisimulation
m is a metric-bisimulation if: m(s, t) < ε⇒:
s −→ P⇒ t −→ Q, m(P,Q) < ε
t −→ Q⇒ s −→ P, m(P,Q) < ε
Problem: what is m(P,Q)? — Type mismatch!!
Need a way to lift distances from states to a distances ondistributions of states.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 26 / 41
![Page 52: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/52.jpg)
Metrics
A putative definition of a metric-bisimulation
m is a metric-bisimulation if: m(s, t) < ε⇒:
s −→ P⇒ t −→ Q, m(P,Q) < ε
t −→ Q⇒ s −→ P, m(P,Q) < ε
Problem: what is m(P,Q)? — Type mismatch!!Need a way to lift distances from states to a distances ondistributions of states.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 26 / 41
![Page 53: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/53.jpg)
Metrics
A detour: Kantorovich metric
Metrics on probability measures on metric spaces.
M: 1-bounded pseudometrics on states.
d(µ, ν) = supf|∫
fdµ−∫
fdν|, f 1-Lipschitz
Arises in the solution of an LP problem: transshipment.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 27 / 41
![Page 54: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/54.jpg)
Metrics
A detour: Kantorovich metric
Metrics on probability measures on metric spaces.M: 1-bounded pseudometrics on states.
d(µ, ν) = supf|∫
fdµ−∫
fdν|, f 1-Lipschitz
Arises in the solution of an LP problem: transshipment.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 27 / 41
![Page 55: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/55.jpg)
Metrics
A detour: Kantorovich metric
Metrics on probability measures on metric spaces.M: 1-bounded pseudometrics on states.
d(µ, ν) = supf|∫
fdµ−∫
fdν|, f 1-Lipschitz
Arises in the solution of an LP problem: transshipment.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 27 / 41
![Page 56: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/56.jpg)
Metrics
A detour: Kantorovich metric
Metrics on probability measures on metric spaces.M: 1-bounded pseudometrics on states.
d(µ, ν) = supf|∫
fdµ−∫
fdν|, f 1-Lipschitz
Arises in the solution of an LP problem: transshipment.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 27 / 41
![Page 57: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/57.jpg)
Metrics
An LP version for Finite-State Spaces
When state space is finite: Let P,Q be probability distributions. Then:
m(P,Q) = max∑
i
(P(si)− Q(si))ai
subject to:∀i.0 ≤ ai ≤ 1∀i, j. ai − aj ≤ m(si, sj).
Panangaden (McGill) Labelled Markov Processes MFPS XXX 28 / 41
![Page 58: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/58.jpg)
Metrics
The Dual Form
Dual form from Worrell and van Breugel:
min∑
i,j
lijm(si, sj) +∑
i
xi +∑
j
yj
subject to:∀i.∑
j lij + xi = P(si)
∀j.∑
i lij + yj = Q(sj)∀i, j. lij, xi, yj ≥ 0.
We prove many equations by using the primal form to show onedirection and the dual to show the other.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 29 / 41
![Page 59: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/59.jpg)
Metrics
Return from Detour
Summary of detour: Given a metric on states in a metric space, can liftto a metric on probability distributions on states.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 30 / 41
![Page 60: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/60.jpg)
Metrics
Metric “Bisimulation”
m is a metric-bisimulation if: m(s, t) < ε⇒:
s −→ P⇒ t −→ Q, m(P,Q) < ε
t −→ Q⇒ s −→ P, m(P,Q) < ε
The required canonical metric on processes is the least such: ie.the distances are the least possible.Thm: Canonical least metric exists. Usual fixed-point theoryarguments.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 31 / 41
![Page 61: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/61.jpg)
Metrics
Metrics: some details
M: 1-bounded pseudometrics on states with ordering
m1 � m2 if (∀s, t) [m1(s, t) ≥ m2(s, t)]
(M,�) is a complete lattice.
⊥(s, t) =
{0 if s = t1 otherwise
>(s, t) = 0, (∀s, t)(u{mi}(s, t) = sup
imi(s, t)
Panangaden (McGill) Labelled Markov Processes MFPS XXX 32 / 41
![Page 62: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/62.jpg)
Metrics
Maximum fixed point definition
Let m ∈M. F(m)(s, t) < ε if:
s −→ P⇒ t −→ Q, m(P,Q) < ε
t −→ Q⇒ s −→ P, m(P,Q) < ε
F(m)(s, t) can be given by an explicit expression.F is monotone onM, and metric-bisimulation is the greatest fixedpoint of F.The closure ordinal of F is ω.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 33 / 41
![Page 63: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/63.jpg)
Metrics
Maximum fixed point definition
Let m ∈M. F(m)(s, t) < ε if:
s −→ P⇒ t −→ Q, m(P,Q) < ε
t −→ Q⇒ s −→ P, m(P,Q) < ε
F(m)(s, t) can be given by an explicit expression.
F is monotone onM, and metric-bisimulation is the greatest fixedpoint of F.The closure ordinal of F is ω.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 33 / 41
![Page 64: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/64.jpg)
Metrics
Maximum fixed point definition
Let m ∈M. F(m)(s, t) < ε if:
s −→ P⇒ t −→ Q, m(P,Q) < ε
t −→ Q⇒ s −→ P, m(P,Q) < ε
F(m)(s, t) can be given by an explicit expression.F is monotone onM, and metric-bisimulation is the greatest fixedpoint of F.
The closure ordinal of F is ω.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 33 / 41
![Page 65: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/65.jpg)
Metrics
Maximum fixed point definition
Let m ∈M. F(m)(s, t) < ε if:
s −→ P⇒ t −→ Q, m(P,Q) < ε
t −→ Q⇒ s −→ P, m(P,Q) < ε
F(m)(s, t) can be given by an explicit expression.F is monotone onM, and metric-bisimulation is the greatest fixedpoint of F.The closure ordinal of F is ω.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 33 / 41
![Page 66: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/66.jpg)
Metrics
A logical metric
Develop a real-valued “modal logic” based on the analogy due toKozen:
Program Logic Probabilistic LogicState s Distribution µFormula φ Random Variable fSatisfaction s |= φ
∫f dµ
Define a metric based on how closely the random variables agree.We did this before the LP based techniques became available.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 34 / 41
![Page 67: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/67.jpg)
Metrics
A logical metric
Develop a real-valued “modal logic” based on the analogy due toKozen:
Program Logic Probabilistic LogicState s Distribution µFormula φ Random Variable fSatisfaction s |= φ
∫f dµ
Define a metric based on how closely the random variables agree.
We did this before the LP based techniques became available.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 34 / 41
![Page 68: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/68.jpg)
Metrics
A logical metric
Develop a real-valued “modal logic” based on the analogy due toKozen:
Program Logic Probabilistic LogicState s Distribution µFormula φ Random Variable fSatisfaction s |= φ
∫f dµ
Define a metric based on how closely the random variables agree.We did this before the LP based techniques became available.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 34 / 41
![Page 69: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/69.jpg)
Metrics
Real-valued Modal Logic
f ::= 1 | max(f , f ) | h ◦ f | 〈a〉.f
1(s) = 1 Truemax(f1, f2)(s) = max(f1(s), f2(s)) Conjunctionh ◦ f (s) = h(f (s)) Lipschitz〈a〉.f (s) = γ
∫s′∈S f (s′)τa(s, ds′) a-transition
where h 1-Lipschitz : [0, 1]→ [0, 1] and γ ∈ (0, 1].d(s, t) = supf |f (s)− f (t)|Thm: d coincides with the canonical metric-bisimulation.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 35 / 41
![Page 70: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/70.jpg)
Metrics
Real-valued Modal Logic
f ::= 1 | max(f , f ) | h ◦ f | 〈a〉.f
1(s) = 1 Truemax(f1, f2)(s) = max(f1(s), f2(s)) Conjunctionh ◦ f (s) = h(f (s)) Lipschitz〈a〉.f (s) = γ
∫s′∈S f (s′)τa(s, ds′) a-transition
where h 1-Lipschitz : [0, 1]→ [0, 1] and γ ∈ (0, 1].
d(s, t) = supf |f (s)− f (t)|Thm: d coincides with the canonical metric-bisimulation.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 35 / 41
![Page 71: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/71.jpg)
Metrics
Real-valued Modal Logic
f ::= 1 | max(f , f ) | h ◦ f | 〈a〉.f
1(s) = 1 Truemax(f1, f2)(s) = max(f1(s), f2(s)) Conjunctionh ◦ f (s) = h(f (s)) Lipschitz〈a〉.f (s) = γ
∫s′∈S f (s′)τa(s, ds′) a-transition
where h 1-Lipschitz : [0, 1]→ [0, 1] and γ ∈ (0, 1].d(s, t) = supf |f (s)− f (t)|
Thm: d coincides with the canonical metric-bisimulation.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 35 / 41
![Page 72: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/72.jpg)
Metrics
Real-valued Modal Logic
f ::= 1 | max(f , f ) | h ◦ f | 〈a〉.f
1(s) = 1 Truemax(f1, f2)(s) = max(f1(s), f2(s)) Conjunctionh ◦ f (s) = h(f (s)) Lipschitz〈a〉.f (s) = γ
∫s′∈S f (s′)τa(s, ds′) a-transition
where h 1-Lipschitz : [0, 1]→ [0, 1] and γ ∈ (0, 1].d(s, t) = supf |f (s)− f (t)|Thm: d coincides with the canonical metric-bisimulation.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 35 / 41
![Page 73: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/73.jpg)
Metrics
The role of γ
γ discounts the value of future steps.
γ < 1 and γ = 1 yield very different topologies.For γ < 1 there is an LP-based strongly-polynomial (in the numberof constraints, and the number of bits of precision required)algorithm to compute the metric.For γ = 1 an algorithm to compute the metric has been discoveredby van Breugel et al.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 36 / 41
![Page 74: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/74.jpg)
Metrics
The role of γ
γ discounts the value of future steps.γ < 1 and γ = 1 yield very different topologies.
For γ < 1 there is an LP-based strongly-polynomial (in the numberof constraints, and the number of bits of precision required)algorithm to compute the metric.For γ = 1 an algorithm to compute the metric has been discoveredby van Breugel et al.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 36 / 41
![Page 75: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/75.jpg)
Metrics
The role of γ
γ discounts the value of future steps.γ < 1 and γ = 1 yield very different topologies.For γ < 1 there is an LP-based strongly-polynomial (in the numberof constraints, and the number of bits of precision required)algorithm to compute the metric.
For γ = 1 an algorithm to compute the metric has been discoveredby van Breugel et al.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 36 / 41
![Page 76: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/76.jpg)
Metrics
The role of γ
γ discounts the value of future steps.γ < 1 and γ = 1 yield very different topologies.For γ < 1 there is an LP-based strongly-polynomial (in the numberof constraints, and the number of bits of precision required)algorithm to compute the metric.For γ = 1 an algorithm to compute the metric has been discoveredby van Breugel et al.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 36 / 41
![Page 77: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/77.jpg)
Approximation
Approximation Results
Our main result is a systematic approximation scheme for labelledMarkov processes. The set of LMPs is a Polish space.
For any LMP, we explicitly provide a (countable) sequence ofapproximants to it such that:
1 For every logical property satisfied by a process, there is anelement of the chain that also satisfies the property.
2 The sequence of approximants converges, in the metric definedbefore, to the process that is being approximated.
The essential idea: approximate bisimulation.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 37 / 41
![Page 78: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/78.jpg)
Approximation
Approximation Results
Our main result is a systematic approximation scheme for labelledMarkov processes. The set of LMPs is a Polish space.For any LMP, we explicitly provide a (countable) sequence ofapproximants to it such that:
1 For every logical property satisfied by a process, there is anelement of the chain that also satisfies the property.
2 The sequence of approximants converges, in the metric definedbefore, to the process that is being approximated.
The essential idea: approximate bisimulation.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 37 / 41
![Page 79: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/79.jpg)
Approximation
Approximation Results
Our main result is a systematic approximation scheme for labelledMarkov processes. The set of LMPs is a Polish space.For any LMP, we explicitly provide a (countable) sequence ofapproximants to it such that:
1 For every logical property satisfied by a process, there is anelement of the chain that also satisfies the property.
2 The sequence of approximants converges, in the metric definedbefore, to the process that is being approximated.
The essential idea: approximate bisimulation.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 37 / 41
![Page 80: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/80.jpg)
Approximation
Domain-theoretic approximation of LMPs
we establish the following equivalence of categories:
LMP ' Proc
where LMP is the category with objects LMPs and withmorphisms simulations; and Proc is the solution to the recursivedomain equation
Proc '∏
Labels
PProb(Proc).
We show that there is a perfect match between:bisimulation and equality in Proc,simulation and the partial order of Proc,strict simulation and way below in Proc.
The sequence of approximants is a directed set in the simulationordering and the process being approximated is the sup of thisdirected set.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 38 / 41
![Page 81: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/81.jpg)
Approximation
Domain-theoretic approximation of LMPs
we establish the following equivalence of categories:
LMP ' Proc
where LMP is the category with objects LMPs and withmorphisms simulations; and Proc is the solution to the recursivedomain equation
Proc '∏
Labels
PProb(Proc).
We show that there is a perfect match between:bisimulation and equality in Proc,simulation and the partial order of Proc,strict simulation and way below in Proc.
The sequence of approximants is a directed set in the simulationordering and the process being approximated is the sup of thisdirected set.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 38 / 41
![Page 82: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/82.jpg)
Approximation
Domain-theoretic approximation of LMPs
we establish the following equivalence of categories:
LMP ' Proc
where LMP is the category with objects LMPs and withmorphisms simulations; and Proc is the solution to the recursivedomain equation
Proc '∏
Labels
PProb(Proc).
We show that there is a perfect match between:bisimulation and equality in Proc,simulation and the partial order of Proc,strict simulation and way below in Proc.
The sequence of approximants is a directed set in the simulationordering and the process being approximated is the sup of thisdirected set.Panangaden (McGill) Labelled Markov Processes MFPS XXX 38 / 41
![Page 83: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/83.jpg)
Approximation
Approximation by averaging
The latest idea is to view LMPs as function transformers.
Functorial view of expectation values.Then bisimulation is naturally dualized and gives eventbisimulation.Approximation is formalized by “coarsening the σ-algebra” ratherthan by clustering points.The approximations form a profinite family that gives thebisimulation-minimal version of the original LMP as a projectivelimit.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 39 / 41
![Page 84: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/84.jpg)
Approximation
Approximation by averaging
The latest idea is to view LMPs as function transformers.Functorial view of expectation values.
Then bisimulation is naturally dualized and gives eventbisimulation.Approximation is formalized by “coarsening the σ-algebra” ratherthan by clustering points.The approximations form a profinite family that gives thebisimulation-minimal version of the original LMP as a projectivelimit.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 39 / 41
![Page 85: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/85.jpg)
Approximation
Approximation by averaging
The latest idea is to view LMPs as function transformers.Functorial view of expectation values.Then bisimulation is naturally dualized and gives eventbisimulation.
Approximation is formalized by “coarsening the σ-algebra” ratherthan by clustering points.The approximations form a profinite family that gives thebisimulation-minimal version of the original LMP as a projectivelimit.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 39 / 41
![Page 86: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/86.jpg)
Approximation
Approximation by averaging
The latest idea is to view LMPs as function transformers.Functorial view of expectation values.Then bisimulation is naturally dualized and gives eventbisimulation.Approximation is formalized by “coarsening the σ-algebra” ratherthan by clustering points.
The approximations form a profinite family that gives thebisimulation-minimal version of the original LMP as a projectivelimit.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 39 / 41
![Page 87: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/87.jpg)
Approximation
Approximation by averaging
The latest idea is to view LMPs as function transformers.Functorial view of expectation values.Then bisimulation is naturally dualized and gives eventbisimulation.Approximation is formalized by “coarsening the σ-algebra” ratherthan by clustering points.The approximations form a profinite family that gives thebisimulation-minimal version of the original LMP as a projectivelimit.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 39 / 41
![Page 88: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/88.jpg)
Approximation
Conclusions
A very fast overview of some of the work on LMPs.
I have skipped the work by Mislove et al. on C∗-algebra duality forLMPs and also on testing equivalences.Also many results by Doberkat, d’Argenio, Varacca,Goubault-Larrecq, Segala, Mio, Simpson, Jacobs, Ying,.....Josée: Logical characterization of bisimulation.Radu: Completeness theorems and duality.Doina: Machine learning.Probabilistic reasoning, modelling and programming is in itsheyday.A major theme of this MFPS: Invited talk and special session pluscontributed talks.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 40 / 41
![Page 89: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/89.jpg)
Approximation
Conclusions
A very fast overview of some of the work on LMPs.I have skipped the work by Mislove et al. on C∗-algebra duality forLMPs and also on testing equivalences.
Also many results by Doberkat, d’Argenio, Varacca,Goubault-Larrecq, Segala, Mio, Simpson, Jacobs, Ying,.....Josée: Logical characterization of bisimulation.Radu: Completeness theorems and duality.Doina: Machine learning.Probabilistic reasoning, modelling and programming is in itsheyday.A major theme of this MFPS: Invited talk and special session pluscontributed talks.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 40 / 41
![Page 90: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/90.jpg)
Approximation
Conclusions
A very fast overview of some of the work on LMPs.I have skipped the work by Mislove et al. on C∗-algebra duality forLMPs and also on testing equivalences.Also many results by Doberkat, d’Argenio, Varacca,Goubault-Larrecq, Segala, Mio, Simpson, Jacobs, Ying,.....
Josée: Logical characterization of bisimulation.Radu: Completeness theorems and duality.Doina: Machine learning.Probabilistic reasoning, modelling and programming is in itsheyday.A major theme of this MFPS: Invited talk and special session pluscontributed talks.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 40 / 41
![Page 91: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/91.jpg)
Approximation
Conclusions
A very fast overview of some of the work on LMPs.I have skipped the work by Mislove et al. on C∗-algebra duality forLMPs and also on testing equivalences.Also many results by Doberkat, d’Argenio, Varacca,Goubault-Larrecq, Segala, Mio, Simpson, Jacobs, Ying,.....Josée: Logical characterization of bisimulation.
Radu: Completeness theorems and duality.Doina: Machine learning.Probabilistic reasoning, modelling and programming is in itsheyday.A major theme of this MFPS: Invited talk and special session pluscontributed talks.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 40 / 41
![Page 92: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/92.jpg)
Approximation
Conclusions
A very fast overview of some of the work on LMPs.I have skipped the work by Mislove et al. on C∗-algebra duality forLMPs and also on testing equivalences.Also many results by Doberkat, d’Argenio, Varacca,Goubault-Larrecq, Segala, Mio, Simpson, Jacobs, Ying,.....Josée: Logical characterization of bisimulation.Radu: Completeness theorems and duality.
Doina: Machine learning.Probabilistic reasoning, modelling and programming is in itsheyday.A major theme of this MFPS: Invited talk and special session pluscontributed talks.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 40 / 41
![Page 93: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/93.jpg)
Approximation
Conclusions
A very fast overview of some of the work on LMPs.I have skipped the work by Mislove et al. on C∗-algebra duality forLMPs and also on testing equivalences.Also many results by Doberkat, d’Argenio, Varacca,Goubault-Larrecq, Segala, Mio, Simpson, Jacobs, Ying,.....Josée: Logical characterization of bisimulation.Radu: Completeness theorems and duality.Doina: Machine learning.
Probabilistic reasoning, modelling and programming is in itsheyday.A major theme of this MFPS: Invited talk and special session pluscontributed talks.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 40 / 41
![Page 94: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/94.jpg)
Approximation
Conclusions
A very fast overview of some of the work on LMPs.I have skipped the work by Mislove et al. on C∗-algebra duality forLMPs and also on testing equivalences.Also many results by Doberkat, d’Argenio, Varacca,Goubault-Larrecq, Segala, Mio, Simpson, Jacobs, Ying,.....Josée: Logical characterization of bisimulation.Radu: Completeness theorems and duality.Doina: Machine learning.Probabilistic reasoning, modelling and programming is in itsheyday.
A major theme of this MFPS: Invited talk and special session pluscontributed talks.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 40 / 41
![Page 95: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/95.jpg)
Approximation
Conclusions
A very fast overview of some of the work on LMPs.I have skipped the work by Mislove et al. on C∗-algebra duality forLMPs and also on testing equivalences.Also many results by Doberkat, d’Argenio, Varacca,Goubault-Larrecq, Segala, Mio, Simpson, Jacobs, Ying,.....Josée: Logical characterization of bisimulation.Radu: Completeness theorems and duality.Doina: Machine learning.Probabilistic reasoning, modelling and programming is in itsheyday.A major theme of this MFPS: Invited talk and special session pluscontributed talks.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 40 / 41
![Page 96: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/96.jpg)
Approximation
Conclusions
A very fast overview of some of the work on LMPs.I have skipped the work by Mislove et al. on C∗-algebra duality forLMPs and also on testing equivalences.Also many results by Doberkat, d’Argenio, Varacca,Goubault-Larrecq, Segala, Mio, Simpson, Jacobs, Ying,.....Josée: Logical characterization of bisimulation.Radu: Completeness theorems and duality.Doina: Machine learning.Probabilistic reasoning, modelling and programming is in itsheyday.A major theme of this MFPS: Invited talk and special session pluscontributed talks.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 40 / 41
![Page 97: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/97.jpg)
Approximation
The End
Thanks for listening!
Buy the book: Labelled Markov Processes Imperial College Press,2009.
Available for free on various pirate websites.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 41 / 41
![Page 98: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/98.jpg)
Approximation
The End
Thanks for listening!
Buy the book: Labelled Markov Processes Imperial College Press,2009.
Available for free on various pirate websites.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 41 / 41
![Page 99: Labelled Markov Processes - cs.cornell.edu](https://reader031.fdocuments.es/reader031/viewer/2022012517/6191dd671339306e6d0c6924/html5/thumbnails/99.jpg)
Approximation
The End
Thanks for listening!
Buy the book: Labelled Markov Processes Imperial College Press,2009.
Available for free on various pirate websites.
Panangaden (McGill) Labelled Markov Processes MFPS XXX 41 / 41