Teach you to build a mobile APM monitoring system

Posted Jun 28, 2020117 min read

The article is nearly 50000 words, sf markdown has a compatibility problem, there is a problem with the format copied from the github md file content, I have made the problematic code into a picture with carbon, and the reading experience may be affected. Originally on github , the reading experience is better, if you are interested, you can move.


APM is short for Application Performance Monitoring and monitors and manages the performance and availability of software applications. Application performance management is critical to the continuous and stable operation of an application. So this article will talk about how to accurately monitor and how to report data from the latitude of performance management of an iOS App.

App performance issues are one of the important factors affecting user experience. Performance issues mainly include:Crash, network request errors or timeouts, slow UI response, main thread stalls, high CPU and memory usage, high power consumption, etc. Most of the problems are caused by developers using thread locks, system functions, programming specifications, data structures, etc. by mistake. The key to solving the problem is to find and locate the problem as soon as possible.

This article focuses on the reasons for APM and how to collect data. After collecting the APM data and combining the data reporting mechanism, upload the data to the server according to a certain strategy. The server consumes this information and produces a report. Please combine Sister Article to summarize how to create a flexible, configurable and powerful Data reporting component.

  1. Caton monitoring

The stuck problem is the problem that the main thread cannot respond to user interaction. Affects the user's direct experience, so the monitoring of app freeze is an important part of APM.

FPS(frame per second) is the best frame refresh rate per second. For iPhones, 60 is the best. For some iPad models, it is 120. It is also a reference parameter for Caton monitoring. Why is it a reference parameter? Because it is not accurate. Let me talk about how to get FPS first. CADisplayLink is a system timer that refreshes the view at the same rate as the frame refresh rate. [CADisplayLink displayLinkWithTarget:self selector:@selector(###:)]. As for why not allow us to take a look at the sample code below

_displayLink = [CADisplayLink displayLinkWithTarget:self selector:@selector(p_displayLinkTick:)];
[_displayLink setPaused:YES];
[_displayLink addToRunLoop:[NSRunLoop currentRunLoop]forMode:NSRunLoopCommonModes];

The code shows that the CADisplayLink object is added to a Mode in the specified RunLoop. Therefore, it is still a CPU-level operation. The stuttering experience is the result of the entire image rendering:CPU + GPU. please watch the following part

1. Principle of screen drawing

Principle of Old CRT Display

Talk about the principle of the old CRT monitor. The CRT electron gun scans line by line from top to bottom according to the above method. After the scan is completed, the display presents a frame of pictures, and then the electron gun returns to the initial position to continue the next scan. In order to synchronize the display process of the display with the video controller of the system, the display(or other hardware) uses the hardware clock to generate a series of timing signals. When the electron gun changes to a new line and is ready to scan, the display will send out a horizontal synchronization signal(horizontal synchronization), or HSync for short; when a frame is drawn, the electron gun returns to its original position, and the monitor is ready to draw the next frame. Will send a vertical synchronization signal(Vertical synchronization), referred to as VSync. The display is usually refreshed at a fixed frequency. This fixed refresh frequency is the frequency generated by the VSync signal. Although the current displays are basically LCD screens, the principle remains the same.

Relationship between display, CPU and GPU

Normally, the display of a picture on the screen is made by the CPU, GPU and display working in the same way as the above picture. The CPU calculates the actual content(such as view creation, layout calculation, picture decoding, text drawing, etc.) based on the code written by the engineer, and then submits the calculation result to the GPU. The GPU is responsible for layer synthesis and texture rendering, and then the GPU will render the result Submit to the frame buffer. Then the video controller will read the data of the frame buffer line by line according to the VSync signal and pass it to the display after digital-to-analog conversion.

In the case where there is only one frame buffer, there are efficiency problems in reading and refreshing the frame buffer. In order to solve the efficiency problem, the display system will introduce two buffers, namely the double buffer mechanism. In this case, the GPU will pre-render a frame and put it in the frame buffer for the video controller to read. When the next frame is rendered, the GPU directly points the video controller's pointer to the second buffer. Increased efficiency.

At present, the double buffer improves efficiency, but brings new problems:when the video controller has not finished reading, that is, the screen content is displayed, the GPU submits the newly rendered frame to another frame Buffer and point the pointer of the video controller to the new frame buffer, the video controller will display the lower half of the new frame of data on the screen, causing the picture torn.

In order to solve this problem, the GPU usually has a mechanism called vertical synchronization signal(V-Sync). When the vertical synchronization signal is turned on, the GPU will wait for the video controller to send the V-Sync signal before rendering and framing a new frame. Update of the buffer. Such several mechanisms have solved the situation of screen tearing and also increased the smoothness of the screen. But requires more computing resources
IPC Wake RunLoop

Answer

Some people may see "When the vertical sync signal is turned on, the GPU will wait until the video controller sends the V-Sync signal before rendering a new frame and updating the frame buffer." Here, I think that the GPU receives the V -Sync only performs a new one-frame rendering and frame buffer update. Is that double-buffer pointless?

Imagine a process in which a display displays the first frame of images and the second frame of images. First, in the case of double buffers, the GPU first renders a frame of image and stores it in the frame buffer, and then lets the video controller's pointer directly directly to this buffer to display the first frame of image. After the content of the first frame image is displayed, the video controller sends a V-Sync signal, and the GPU renders the second frame image after receiving the V-Sync signal and points the video controller pointer to the second frame buffer.

It seems that the second frame of image is waiting for the V-Sync signal sent by the video controller after the first frame is displayed. is it? Is this really the case? ? What do you want, of course not? ? Otherwise, the double buffer will be meaningless

Secret. Please see the picture below

Multiple buffer display principle

When the first V-Sync signal arrives, first render a frame of image and put it in the frame buffer, but it is not displayed. When the second V-Sync signal is received, read the first rendered result(video control The pointer of the device points to the first frame buffer), and simultaneously renders a new frame of image and stores the result in the second frame buffer. After receiving the third V-Sync signal, read the second frame The content of the buffer(the pointer of the video controller points to the second frame buffer), and starts the rendering of the third frame of the image and sends it to the first frame buffer, and then continuously loops back and forth.
Please check the information: Multiple buffering

2. Causes of Caton

Cause of Stuck

After the VSync signal arrives, the system graphics service will notify the app through mechanisms such as CADisplayLink, and the main thread of the app starts to calculate the display content in the CPU(view creation, layout calculation, picture decoding, text drawing, etc.). Then the calculated content is submitted to the GPU. The GPU performs layer transformation, synthesis, and rendering, and then the GPU submits the rendering result to the frame buffer, and waits for the next VSync signal to display the previously rendered result. In the case of the vertical synchronization mechanism, if the CPU or GPU does not complete the submission of the content within a VSync time period, it will cause the frame to be discarded and wait for the next opportunity to display it. At this time, the screen is still the previously rendered image. So this is the reason why the interface of the CPU and GPU is stuck.

Currently, iOS devices have a double-buffering mechanism and a triple-buffering mechanism. Android currently has a triple-buffering mechanism in the mainstream, and a single-buffering mechanism in the early days.
iOS three buffer mechanism example

There are many reasons for CPU and GPU resource consumption, such as frequent object creation, attribute adjustment, file reading, view level adjustment, layout calculation(AutoLayout view number is greater, it is more difficult to solve linear equations), picture decoding(large picture Reading optimization), image drawing, text rendering, database reading(multiple reading or more writing optimistic locking, pessimistic locking scenarios), the use of locks(for example:improper use of spin locks will waste CPU) and other aspects. Developers find the optimal solution based on their own experience(this is not the focus of this article).

3. How does APM monitor the lag and report

CADisplayLink is definitely not needed, this FPS is just for reference. Generally speaking, there are two options for monitoring Caton:Monitoring RunLoop status callback, sub-thread ping main thread

3.1 RunLoop status monitoring method

RunLoop is responsible for monitoring the input source for scheduling processing. Such as networks, input devices, periodic or delayed events, asynchronous callbacks, etc. RunLoop will receive 2 types of input sources:one is an asynchronous message(source0 event) from another thread or from a different application, and the other is an event from a predetermined or repeated interval.

RunLoop status as shown below
RunLoop

The first step:notify Observers that RunLoop should start to enter the loop, and then enter the loop

if(currentMode->_observerMask & kCFRunLoopEntry)
    //Notify Observers:RunLoop is about to enter the loop
    __CFRunLoopDoObservers(rl, currentMode, kCFRunLoopEntry);
//enter the loop
result = __CFRunLoopRun(rl, currentMode, seconds, returnAfterSourceHandled, previousMode);

The second step:open the do while loop to keep the thread alive, notify Observers, RunLoop triggers the timer callback, Source0 callback, and then executes the added block

 if(rlm->_observerMask & kCFRunLoopBeforeTimers)
    //Notify Observers:RunLoop will trigger Timer callback soon
    __CFRunLoopDoObservers(rl, rlm, kCFRunLoopBeforeTimers);
if(rlm->_observerMask & kCFRunLoopBeforeSources)
    //Notify Observers:RunLoop is about to trigger a source callback
    __CFRunLoopDoObservers(rl, rlm, kCFRunLoopBeforeSources);
//execute the added block
__CFRunLoopDoBlocks(rl, rlm);

Step 3:After RunLoop triggers the Source0 callback, if Source1 is ready, it will jump to handle_msg to process the message.

//If Source1(based on port) is in the ready state, directly process this Source1 and then jump to process the message
if(MACH_PORT_NULL != dispatchPort && !didDispatchPortLastTime) {
#if DEPLOYMENT_TARGET_MACOSX || DEPLOYMENT_TARGET_EMBEDDED || DEPLOYMENT_TARGET_EMBEDDED_MINI
    msg =(mach_msg_header_t *)msg_buffer;

    if(__CFRunLoopServiceMachPort(dispatchPort, &msg, sizeof(msg_buffer), &livePort, 0, &voucherState, NULL)) {
        goto handle_msg;
    }
#elif DEPLOYMENT_TARGET_WINDOWS
    if(__CFRunLoopWaitForMultipleObjects(NULL, &dispatchPort, 0, 0, &livePort, NULL)) {
        goto handle_msg;
    }
#endif
}

Step 4:After the callback is triggered, notify Observers that it is about to go to sleep

Boolean poll = sourceHandledThisLoop ||(0ULL == timeout_context->termTSR);
//Notify Observers:RunLoop thread is about to go to sleep(sleep)
if(!poll &&(rlm->_observerMask & kCFRunLoopBeforeWaiting)) __CFRunLoopDoObservers(rl, rlm, kCFRunLoopBeforeWaiting);
    __CFRunLoopSetSleeping(rl);

Step 5:After entering sleep, it will wait for mach_port message to wake up again. Only the following 4 conditions can be awakened again.

  • Source event based on port

  • Timer is up

  • RunLoop timeout

  • The callee wakes up

    do {

      if(kCFUseCollectableAllocator) {
          //objc_clear_stack(0);
          //<rdar://problem/16393959>
          memset(msg_buffer, 0, sizeof(msg_buffer));
      }
      msg =(mach_msg_header_t *)msg_buffer;
    
      __CFRunLoopServiceMachPort(waitSet, &msg, sizeof(msg_buffer), &livePort, poll? 0:TIMEOUT_INFINITY, &voucherState, &voucherCopy);
    
      if(modeQueuePort != MACH_PORT_NULL && livePort == modeQueuePort) {
          //Drain the internal queue. If one of the callout blocks sets the timerFired flag, break out and service the timer.
          while(_dispatch_runloop_root_queue_perform_4CF(rlm->_queue));
          if(rlm->_timerFired) {
              //Leave livePort as the queue port, and service timers below
              rlm->_timerFired = false;
              break;
          } else {
              if(msg && msg !=(mach_msg_header_t *)msg_buffer) free(msg);
          }
      } else {
          //Go ahead and leave the inner loop.
          break;
      }

    } while(1);

Step 6:Notify Observer when awakening, the RunLoop thread has just been awakened

//Notify Observers:The RunLoop thread has just been woken up
if(!poll &&(rlm->_observerMask & kCFRunLoopAfterWaiting)) __CFRunLoopDoObservers(rl, rlm, kCFRunLoopAfterWaiting);
    //Process the message
    handle_msg:;
    __CFRunLoopSetIgnoreWakeUps(rl);

Step 7:After the RunLoop wakes up, process the message received at the wake up

  • If it is the Timer time, trigger the callback of Timer

  • If it is dispatch, then execute block

  • If it is a source1 event, then handle this event

    #if USE_MK_TIMER_TOO

          //If a Timer is up, trigger the Timer's callback
          else if(rlm->_timerPort != MACH_PORT_NULL && livePort == rlm->_timerPort) {
              CFRUNLOOP_WAKEUP_FOR_TIMER();
              //On Windows, we have observed an issue where the timer port is set before the time which we requested it to be set. For example, we set the fire time to be TSR 167646765860, but it is actually observed firing at TSR 167646764145, which is 1715 ticks early. The result is that, when __CFRunLoopDoTimers checks to see if any of the run loop timers should be firing, it appears to be'too early' for the next timer, and no timers are handled.
              //In this case, the timer port has been automatically reset(since it was returned from MsgWaitForMultipleObjectsEx), and if we do not re-arm it, then no timers will ever be serviced again unless something adjusts the timer list(eg adding or removing timers). The fix for the issue is to reset the timer here if CFRunLoopDoTimers did not handle a timer itself. 9308754
              if(!__CFRunLoopDoTimers(rl, rlm, mach_absolute_time())) {
                  //Re-arm the next timer
                  __CFArmNextTimerInMode(rlm, rl);
              }
          }

    #endif

          //If there is a block dispatched to main_queue, execute the block
          else if(livePort == dispatchPort) {
              CFRUNLOOP_WAKEUP_FOR_DISPATCH();
              __CFRunLoopModeUnlock(rlm);
              __CFRunLoopUnlock(rl);
              _CFSetTSD(__CFTSDKeyIsInGCDMainQ,(void *)6, NULL);

    #if DEPLOYMENT_TARGET_WINDOWS

              void *msg = 0;

    #endif

              __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__(msg);
              _CFSetTSD(__CFTSDKeyIsInGCDMainQ,(void *)0, NULL);
              __CFRunLoopLock(rl);
              __CFRunLoopModeLock(rlm);
              sourceHandledThisLoop = true;
              didDispatchPortLastTime = true;
          }
          //If a Source1(based on port) emits an event, process the event
          else {
              CFRUNLOOP_WAKEUP_FOR_SOURCE();
    
              //If we received a voucher from this mach_msg, then put a copy of the new voucher into TSD. CFMachPortBoost will look in the TSD for the voucher. By using the value in the TSD we tie the CFMachPortBoost to this received mach_msg explicitly without a chance for anything in between the two pieces of code to set the voucher again.
              voucher_t previousVoucher = _CFSetTSD(__CFTSDKeyMachMessageHasVoucher,(void *)voucherCopy, os_release);
    
              CFRunLoopSourceRef rls = __CFRunLoopModeFindSourceForMachPort(rl, rlm, livePort);
              if(rls) {

    #if DEPLOYMENT_TARGET_MACOSX || DEPLOYMENT_TARGET_EMBEDDED || DEPLOYMENT_TARGET_EMBEDDED_MINI

          mach_msg_header_t *reply = NULL;
          sourceHandledThisLoop = __CFRunLoopDoSource1(rl, rlm, rls, msg, msg->msgh_size, &reply) || sourceHandledThisLoop;
          if(NULL != reply) {
             (void)mach_msg(reply, MACH_SEND_MSG, reply->msgh_size, 0, MACH_PORT_NULL, 0, MACH_PORT_NULL);
              CFAllocatorDeallocate(kCFAllocatorSystemDefault, reply);
          }

    #elif DEPLOYMENT_TARGET_WINDOWS

                  sourceHandledThisLoop = __CFRunLoopDoSource1(rl, rlm, rls) || sourceHandledThisLoop;

    #endif

Step 8:Determine whether to enter the next loop according to the current RunLoop state. When it is forcibly stopped by the outside or the loop times out, the next loop is not continued, otherwise the next loop is entered

if(sourceHandledThisLoop && stopAfterHandle) {
    //When entering the loop, the parameter says to return after processing the event
    retVal = kCFRunLoopRunHandledSource;
    } else if(timeout_context->termTSR <mach_absolute_time()) {
        //The timeout period of the incoming parameter marker is exceeded
        retVal = kCFRunLoopRunTimedOut;
} else if(__CFRunLoopIsStopped(rl)) {
        __CFRunLoopUnsetStopped(rl);
    //Forcibly stopped by external caller
    retVal = kCFRunLoopRunStopped;
} else if(rlm->_stopped) {
    rlm->_stopped = false;
    retVal = kCFRunLoopRunStopped;
} else if(__CFRunLoopModeIsEmpty(rl, rlm, previousMode)) {
    //No source/timer
    retVal = kCFRunLoopRunFinished;
}

The complete and annotated RunLoop code can be found here [ https://raw.githubusercontent.com/FantasticLBP/knowledge-kit/master/assets/CFRunLoop.c ). Source1 is used by RunLoop to process system events from Mach port, and Source0 is used to process user events. After receiving the system event of Source1, the essence is to call the processing function of Source0 event.

RunLoop Status
RunLoop 6 states

typedef CF_OPTIONS(CFOptionFlags, CFRunLoopActivity) {
    kCFRunLoopEntry, //enter loop
    kCFRunLoopBeforeTimers, //Trigger Timer callback
    kCFRunLoopBeforeSources, //trigger Source0 callback
    kCFRunLoopBeforeWaiting, //wait for mach_port message
    kCFRunLoopAfterWaiting), //Receive mach_port message
    kCFRunLoopExit, //exit loop
    kCFRunLoopAllActivities //loop all state changes
}

RunLoop's method execution time before going to sleep is too long to be able to go to sleep, or the thread is too long to receive a message after waking up to enter the next step, it will block the thread. If it is the main thread, it appears to be stuck.

Once it is found that the state of KCFRunLoopBeforeSources before going to sleep, or KCFRunLoopAfterWaiting after wake-up, there is no change within the set time threshold, it can be judged as stuck, then dump the stack information, restore the scene of the case, and then solve the stuck problem.

Start a sub-thread, and continuously monitor whether it is stuck. After n times have exceeded the stutter threshold, it is considered stuttered. After stuttering, perform a stack dump and report(with a certain mechanism, data processing in the next part).

WatchDog has different values in different states.

  • Launch(Launch):20s
  • Resume:10s
  • Suspend:10s
  • Quit:6s
  • Background(Background):3min(can apply for 10min before iOS7; change to 3min afterwards; can apply continuously, up to 10min)

The basis for setting the stall threshold is the mechanism of WatchDog. The threshold in the APM system needs to be less than the value of WatchDog, so the value range is between [1, 6], and the industry usually chooses 3 seconds.

Determine whether to block the main thread by long dispatch_semaphore_wait(dispatch_semaphore_t dsema, dispatch_time_t timeout) method, Returns zero on success, or non-zero if the timeout occurred. Returning non-zero means that the main thread is blocked by timeout.

RunLoop-ANR

Many people may wonder about the status of RunLoop so much, why choose KCFRunLoopBeforeSources and KCFRunLoopAfterWaiting? Because most of the stalls are between KCFRunLoopBeforeSources and KCFRunLoopAfterWaiting. For example, the internal events of the Source0 type app, etc.

Runloop detection stuck flow chart is as follows:
RunLoop ANR

The key code is as follows:

//Set the running environment of Runloop observer
CFRunLoopObserverContext context = {0,(__bridge void *)self, NULL, NULL};
//Create Runloop observer object
_observer = CFRunLoopObserverCreate(kCFAllocatorDefault,
                                    kCFRunLoopAllActivities,
                                    YES,
                                    0,
                                    &runLoopObserverCallBack,
                                    &context);
//Add the new observer to the runloop of the current thread
CFRunLoopAddObserver(CFRunLoopGetMain(), _observer, kCFRunLoopCommonModes);
//create signal
_semaphore = dispatch_semaphore_create(0);
__weak __typeof(self) weakSelf = self;
//Monitoring time in child thread
dispatch_async(dispatch_get_global_queue(0, 0), ^{
    __strong __typeof(weakSelf) strongSelf = weakSelf;
    if(!strongSelf) {
        return;
    }
    while(YES) {
        if(strongSelf.isCancel) {
            return;
        }
        //N times of stuttering exceeding the threshold T is recorded as one stuttering
        long semaphoreWait = dispatch_semaphore_wait(self->_semaphore, dispatch_time(DISPATCH_TIME_NOW, strongSelf.limitMillisecond * NSEC_PER_MSEC));
        if(semaphoreWait != 0) {
            if(self->_activity == kCFRunLoopBeforeSources || self->_activity == kCFRunLoopAfterWaiting) {
                if(++strongSelf.countTime <strongSelf.standstillCount){
                    continue;
                }
                //Stack information dump, combined with data reporting mechanism, upload data to the server according to a certain strategy. Stack dump will be explained below. Data reporting will be in [Building a powerful, flexible and configurable data reporting component](https://github.com/FantasticLBP/knowledge-kit/blob/master/Chapter1%20-%20iOS/1.80.md)
            }
        }
        strongSelf.countTime = 0;
    }
});

3.2 How the child thread pings the main thread

Start a child thread, create a semaphore with an initial value of 0, and a Boolean type flag with an initial value of YES. The task with the flag set to NO is dispatched to the main thread, and the child thread sleeps for a threshold time. After the time expires, it is judged whether the flag bit is successful by the main thread(the value is NO). If it is not successful, the pig thread is considered to be stuck. At this time, dump the stack information and combine the data reporting mechanism to upload data to the server according to a certain strategy. Data reporting will be in Building a powerful, flexible and configurable data reporting component

while(self.isCancelled == NO) {
        @autoreleasepool {
            __block BOOL isMainThreadNoRespond = YES;

            dispatch_semaphore_t semaphore = dispatch_semaphore_create(0);

            dispatch_async(dispatch_get_main_queue(), ^{
                isMainThreadNoRespond = NO;
                dispatch_semaphore_signal(semaphore);
            });

            [NSThread sleepForTimeInterval:self.threshold];

            if(isMainThreadNoRespond) {
                if(self.handlerBlock) {
                    self.handlerBlock(); //Externally dump the stack inside the block(described below), data reporting
                }
            }

            dispatch_semaphore_wait(semaphore, DISPATCH_TIME_FOREVER);
        }
    }

4. Stack dump

Getting the method stack is a hassle. Organize your thinking. [NSThread callStackSymbols] can get the call stack of the current thread. However, when it is detected that stuttering occurs, it is impossible to get the stack information of the main thread. The path from any thread back to the main thread does not work. Do a knowledge review first.

In computer science, the call stack is a stack-type data structure used to store thread information about computer programs. This stack is also called the execution stack, program stack, control stack, runtime stack, machine stack, etc. The call stack is used to keep track of each active subroutine that should return to the point of control after completing execution.

Wikipedia searched a picture and example of "Call Stack", as follows
Call Stack
The figure above is represented as a stack. Divided into several stack frames(Frame), each stack frame corresponds to a function call. The blue part below represents the DrawSquare function, which calls the DrawLine function during execution, indicated by the green part.

It can be seen that the stack frame is composed of three parts:function parameters, return address, and local variables. For example, the DrawLine function is called inside DrawSquare:the first is to put the parameters required by the DrawLine function on the stack; the second is the return address(control information. For example:the function B is called in function A, and the address of the next line of code in function B is returned Address) into the stack; local variables inside the third function are also stored in the stack.

Stack pointer Stack Pointer represents the top of the current stack, most of the operating system is the stack grows downward, so the stack pointer is the minimum value. The address pointed by the frame pointer Frame Pointer stores the value of the last Stack Pointer, that is, the return address.

In most operating systems, each stack frame also saves the frame pointer of the previous stack frame. Therefore, knowing the Stack Pointer and Frame Pointer of the current stack frame can continuously backtrack and recursively obtain the frame at the bottom of the stack.

The next step is to get Stack Pointer and Frame Pointer of all threads. Then keep going back to restore the scene of the crime.

5. Mach Task Knowledge

Mach task:

When the app is running, it will correspond to a Mach Task, and there may be multiple threads executing tasks at the same time under the Task. The "Mach Task" described in "OS X and iOS Kernel Programming" is:A task is a container object. Virtual memory space and other resources are managed through this container object. These resources include devices and other handles. Simply summarized as:Mack task is a machine independent thread execution environment abstraction.

Function:task can be understood as a process, including its thread list.

Structure:task_threads, save all threads under the target_task task in the act_list array, the number of the array is act_listCnt

kern_return_t task_threads

(
task_t traget_task,
thread_act_array_t *act_list, //Thread pointer list
mach_msg_type_number_t *act_listCnt //number of threads
)

thread_info:

kern_return_t thread_info

(
thread_act_t target_act,
thread_flavor_t flavor,
thread_info_t thread_info_out,
mach_msg_type_number_t *thread_info_outCnt
);

How to get the stack data of a thread:

System method kern_return_t task_threads(task_inspect_t target_task, thread_act_array_t *act_list, mach_msg_type_number_t *act_listCnt); can get all threads, but the thread information obtained by this method is the lowest mach thread.

For each thread, you can use kern_return_t thread_get_state(thread_act_t target_act, thread_state_flavor_t flavor, thread_state_t old_state, mach_msg_type_number_t *old_stateCnt); method to get all its information, the information is filled in the parameters of type _STRUCT_MCONTEXT, there are 2 in this method The parameters vary with the CPU architecture. Therefore, it is necessary to define a macro to shield the difference between different CPUs.

The _STRUCT_MCONTEXT structure stores the Stack Pointer of the current thread and the Frame pointer of the top stack frame, and then traces back the entire thread call stack.

But the above method gets the kernel thread, the information we need is NSThread, so we need to convert the kernel thread to NSThread.

The p of pthread is the abbreviation of POSIX, which means "Portable Operating System Interface". The original design intention is that each system has its own unique threading model, and different systems have different APIs for threading operations. So the purpose of POSIX is to provide abstract pthreads and related APIs. These APIs have different implementations in different operating systems, but the completed functions are the same.

The task_threads and thread_get_state provided by Unix systems operate on the kernel system, and each kernel thread is uniquely identified by an id of type thread_t. The unique identifier of pthread is pthread_t type. The conversion of kernel thread and pthread(that is, thread_t and pthread_t) is easy, because the original intention of pthread design is "abstract kernel thread".

The callback function for the thread created by the memorystatus_action_neededpthread_create method is nsthreadLauncher.

static void *nsthreadLauncher(void* thread)
{
    NSThread *t =(NSThread*)thread;
    [nc postNotificationName:NSThreadDidStartNotification object:t userInfo:nil];
    [t _setName:[t name]];
    [t main];
    [NSThread exit];
    return NULL;
}

NSThreadDidStartNotification is actually the string @"_NSThreadDidStartNotification".

<NSThread:0x...>{number = 1, name = main}

In order to correspond to NSThread and kernel thread, only one-to-one correspondence can be made by name. The pthread API pthread_getname_np can also get the kernel thread name. np stands for not POSIX, so it cannot be used across platforms.

The idea is summarized as follows:store the original name of NSThread, and then change the name to a random number(timestamp), and then traverse the name of the kernel thread pthread. When the name matches, NSThread corresponds to the kernel thread. After finding it, restore the name of the thread to the original name. For the main thread, since pthread_getname_np cannot be used, thread_t is obtained in the load method of the current code, and then the name is matched.

static mach_port_t main_thread_id;
+(void)load {
    main_thread_id = mach_thread_self();
}
  1. App startup time monitoring

1. App startup time monitoring

Application startup time is one of the important factors that affect the user experience, so we need to quantify how fast an app starts. Startup is divided into cold start and hot start.
App startup time

Cold start:App is not running yet, you must load and build the entire application. Complete application initialization. There is a large optimization space for cold start. The cold start time is calculated from the application:didFinishLaunchingWithOptions: method, and the App generally performs basic initialization of various SDKs and Apps here.

Hot start:The application is already running in the background(common scenarios:for example, when the user clicks the Home button while using the App and then opens the App), due to certain events that wake the application to the foreground, the App will accept the application to enter the foreground in the applicationWillEnterForeground: method event

The idea is relatively simple. as follows

  • Get the current time value first in the monitoring method's load method
  • Listen for notifications after the app has been launched UIApplicationDidFinishLaunchingNotification
  • Get the current time after receiving the notification
  • The time difference between steps 1 and 3 is the app launch time.

mach_absolute_time is a CPU/bus dependent function that returns a number of CPU clock cycles. It will not increase while the system is sleeping. It is a number in the nanosecond level. It needs to be converted to seconds after 2 nanoseconds before and after acquisition. It needs a benchmark based on the system time, obtained through mach_timebase_info.

mach_timebase_info_data_t g_cmmStartupMonitorTimebaseInfoData = 0;
mach_timebase_info(&g_cmmStartupMonitorTimebaseInfoData);
uint64_t timelapse = mach_absolute_time()-g_cmmLoadTime;
double timeSpan =(timelapse * g_cmmStartupMonitorTimebaseInfoData.numer)/(g_cmmStartupMonitorTimebaseInfoData.denom * 1e9);

2. Online monitoring startup time is good, but the startup time needs to be optimized during the development phase.

To optimize the start-up time, you must first know what was done during the start-up phase and make a plan based on the status quo.

The pre-main phase is defined as the phase from when the App starts to the system call main function; the main phase is defined as the viewDidAppear from the main function entrance to the main UI framework.

App startup process:

  • Parse Info.plist:load related information such as splash screen; sandbox establishment, permission check;
  • Mach-O loading:If it is a fat binary file, find the part suitable for the current CPU architecture; load all dependent Mach-O files(recursively call the method of Mach-O loading); define internal and external pointer references, such as strings, Functions, etc.; methods in loading classification; c++ static object loading, calling Objc's +load() function; executing the c function declared as __attribute_((constructor));
  • Program execution:call main(); call UIApplicationMain(); call applicationWillFinishLaunching();

Pre-Main stage
Pre-Main stage

Main stage
Main stage

2.1 Load Dylib

Each dynamic library is loaded, dyld needs

  • Analysis of dynamic libraries
  • Find the Mach-O file of the dynamic library
  • open a file
  • Verification documents
  • Register the file signature in the core of the system
  • Call mmap() for each segment of the dynamic library

optimization:

  • Reduce the dependence of non-system libraries
  • Use static library instead of dynamic library
  • Merging non-system dynamic libraries into a dynamic library

2.2 Rebase && Binding

optimization:

  • Reduce the number of Objc classes, reduce the number of selectors, and delete unused classes and functions
  • Reduce the number of C++ virtual functions
  • Switch to Swift struct(essentially, reduce the number of symbols)

2.3 Initializers

optimization:

  • Use +initialize instead of +load
  • Don't use attribute*((constructor)) to mark the method display as an initializer, but only to execute it when the initialization method is called. For example, use dispatch_one, pthread_once() or std::once(). c++

2.4 pre-main

  • ObjC
  • C constructor
  • C++
  • ObjC +load
  • framework optional required framework App iOS required optional optional

  • OC AppCode linkmap

    FUI C++

  • +load +initialize C++ ( )

  • iOS __cstring

    Object-c / / Object-c /

  • dispatch_once() attribute((constructor)) C++ ObjC +load

 IO                           TinyPNG 

2.5 main

  • CPU

  • xib storyboard UI UI TabBarController xib storyboard

    CPU


1. CPU

CPU Central Processing Unit ARM arm64 Intel x86 AMD Intel CISC Complex Instruction Set Computer ARM RISC Reduced Instruction Set Computer ** CPU **

CPU CISC ** ** CISC CPU MUL ADDRA, ADDRB ADDRA ADDRB ADDRA ADDRA ADDRB CPU *CISC CPU CPU *

RISC MOVE A, ADDRA; MOVE B, ADDRB; MUL A, B; STR ADDRA, A; CPU CPU

      iPhone      arm64       arm       

2.

        CPU       
  • task task
  • CPU
  • dump
struct thread_basic_info {
    time_value_t    user_time;      /* user run time         */
    time_value_t    system_time;    /* system run time         */
    integer_t       cpu_usage;      /* scaled cpu usage percentage CPU      1000  */
    policy_t        policy;         /* scheduling policy in effect         */
    integer_t       run_state;      /* run state(     ) */
    integer_t       flags;          /* various flags(     ) */
    integer_t       suspend_count;  /* suspend count for thread         */
    integer_t       sleep_time;     /* number of seconds that thread
                                     *  has been sleeping       */
};



thread_act_array_t threads;
mach_msg_type_number_t threadCount = 0;
const task_t thisTask = mach_task_self();
kern_return_t kr = task_threads(thisTask, &threads, &threadCount);
if(kr != KERN_SUCCESS) {
    return;
}
for(int i = 0; i < threadCount; i++) {
    thread_info_data_t threadInfo;
    thread_basic_info_t threadBaseInfo;
    mach_msg_type_number_t threadInfoCount;

    kern_return_t kr = thread_info((thread_inspect_t)threads[i], THREAD_BASIC_INFO,(thread_info_t)threadInfo, &threadInfoCount);

    if(kr == KERN_SUCCESS) {

        threadBaseInfo =(thread_basic_info_t)threadInfo;
        //todo          
        if(!(threadBaseInfo->flags & TH_FLAGS_IDLE)) {
            integer_t cpuUsage = threadBaseInfo->cpu_usage/10;
            if(cpuUsage > CPUMONITORRATE) {

                NSMutableDictionary *CPUMetaDictionary = [NSMutableDictionary dictionary];
                NSData *CPUPayloadData = [NSData data];

                NSString *backtraceOfAllThread = [BacktraceLogger backtraceOfAllThread];
                //1.       Meta   
                CPUMetaDictionary[@"MONITOR_TYPE"]= CMMonitorCPUType;

                //2.       Payload      JSON       Key       STACK_TRACE  value   base64        
                NSData *CPUData = [SAFE_STRING(backtraceOfAllThread) dataUsingEncoding:NSUTF8StringEncoding];
                NSString *CPUDataBase64String = [CPUData base64EncodedStringWithOptions:0];
                NSDictionary *CPUPayloadDictionary = @{@"STACK_TRACE":SAFE_STRING(CPUDataBase64String)};

                NSError *error;
                //NSJSONWritingOptions       0           \n         0      json     \n
                NSData *parsedData = [NSJSONSerialization dataWithJSONObject:CPUPayloadDictionary options:0 error:&error];
                if(error) {
                    CMMLog(@"%@", error);
                    return;
                }
                CPUPayloadData = [parsedData copy];

                //3.        [                 ](https://github.com/FantasticLBP/knowledge-kit/blob/master/Chapter1%20-%20iOS/1.80.md)  
                [[PrismClient sharedInstance]sendWithType:CMMonitorCPUType meta:CPUMetaDictionary payload:CPUPayloadData]; 
            }
        }
    }
}

OOM

1.

             CPU                                  CPU                           CPU                                                                CPU                   CPU                         CPU 

** ** Windows Linux/Unix

iOS iOS **** ** **

2. iOS

RAM CPU iOS

OOM   out-of-memory                     FOOM foreground OOM   BOOM background OOM      iOS   `Jetsam`            Crash       Signal           

Jetsam    Jetsam                                 Jetsam                                        Jetsam           

  Jetsam                                       App             iOS                   Jetsam           App         iOS            App             crash 

2 OOM App App "highg water mark" App App

 xnu/bsd/kern/kern\_memorystatus.c          2      

highwater -> App

  1. p_memstat_memlimit
  2. DiagonoseActive FREEZE
  3. exit

memorystatus_act_aggressive ->

  1. policy jld_bucket_count
  2. JETSAM_PRIORITY_ELEVATED_INACTIVE
  3. Old_bucket_count memorystatus_jld_eval_period_msecs
  4. memorystatus_avail_pages_below_pressure
  • App App App App
  • App App App
  • App App App App App App
  • App

App page out page in

Memory page** page page 16KB 3 page
 page

  • Clean Memory
    Clean memory 3 page out App framework framework _DATA_CONST clean runtime swizling dirty

       page                      App          dirty                  clean page   

    2020-02-28-iOSMemoryTypeClean.png

  • Dirty Memory

    Dirty memory 4 App framework framework _DATA _DATA_DIRTY dirty

    framework         Dirty memory                      Dirty memory                            Dirty memory    

    2020-02-28-iOSMemoryTypeDirty.png

  • Compressed Memory

            iOS              iOS7     **memory compressor**                                                page                                 

    App Framework NSDictionary 3 pages memory compressor 1 page 3 pages

App = pageNumbers * pageSize Compressed Memory Dirty memory Memory footprint = dirtySize + CompressedSize

          App      extension           crash   `EXC_RESOURCE_EXCEPTION`   

Memory footprint

                  App              

3.

3.1 JetsamEvent

App Jetsam Settings-Privacy-Analytics & Improvements- Analytics Data - - - JetsamEvent-2020-03-14-161828.ips JetsamEvent JetsamEvent iOS idle frontmost suspended App

  App                    `pageSize`        per-process-limit           `rpages`    rpages \* pageSize      OOM     

largestProcess      App    reason          states         App      idle suspended frontmost...  

           2    iPhone 6s plus/13.3.1 iPhone 11 Pro/13.3.1     App                      Demo App         ViewController     

-(void)viewDidLoad {
    [super viewDidLoad];
    NSMutableArray *array = [NSMutableArray array];
    for(NSInteger index = 0; index < 10000000; index++) {
        UIImageView *imageView = [[UIImageView alloc]initWithFrame:CGRectMake(0, 0, 100, 100)];
        UIImage *image = [UIImage imageNamed:@"AppIcon"];
        imageView.image = image;
        [array addObject:imageView];
    }
}

iPhone 6s plus/13.3.1

{"bug_type":"298","timestamp":"2020-03-19 17:23:45.94 +0800","os_version":"iPhone OS 13.3.1(17D50)","incident_id":"DA8AF66D-24E8-458C-8734-981866942168"}
{
  "crashReporterKey" :"fc9b659ce486df1ed1b8062d5c7c977a7eb8c851",
  "kernel" :"Darwin Kernel Version 19.3.0:Thu Jan  9 21:10:44 PST 2020; root:xnu-6153.82.3~1\/RELEASE_ARM64_S8000",
  "product" :"iPhone8,2",
  "incident" :"DA8AF66D-24E8-458C-8734-981866942168",
  "date" :"2020-03-19 17:23:45.93 +0800",
  "build" :"iPhone OS 13.3.1(17D50)",
  "timeDelta" :332,
  "memoryStatus" :{
  "compressorSize" :48499,
  "compressions" :7458651,
  "decompressions" :5190200,
  "zoneMapCap" :744407040,
  "largestZone" :"APFS_4K_OBJS",
  "largestZoneSize" :41402368,
  "pageSize" :16384,
  "uncompressed" :104065,
  "zoneMapSize" :141606912,
  "memoryPages" :{
    "active" :26214,
    "throttled" :0,
    "fileBacked" :14903,
    "wired" :20019,
    "anonymous" :37140,
    "purgeable" :142,
    "inactive" :23669,
    "free" :2967,
    "speculative" :2160
  }
},
  "largestProcess" :"Test",
  "genCounter" :0,
  "processes" :[
  {
    "uuid" :"39c5738b-b321-3865-a731-68064c4f7a6f",
    "states" :[
      "daemon",
      "idle"
   ],
    "lifetimeMax" :188,
    "age" :948223699030,
    "purgeable" :0,
    "fds" :25,
    "coalition" :422,
    "rpages" :177,
    "pid" :282,
    "idleDelta" :824711280,
    "name" :"com.apple.Safari.SafeBrowsing.Se",
    "cpuTime" :10.275422000000001
  },
  //...
  {
    "uuid" :"83dbf121-7c0c-3ab5-9b66-77ee926e1561",
    "states" :[
      "frontmost"
   ],
    "killDelta" :2592,
    "genCount" :0,
    "age" :1531004794,
    "purgeable" :0,
    "fds" :50,
    "coalition" :1047,
    "rpages" :92806,
    "reason" :"per-process-limit",
    "pid" :2384,
    "cpuTime" :59.464373999999999,
    "name" :"Test",
    "lifetimeMax" :92806
  },
  //...
]
}

iPhone 6s plus/13.3.1 OOM (16384*92806)/(1024*1024)=1450.09375M

iPhone 11 Pro/13.3.1

{"bug_type":"298","timestamp":"2020-03-19 17:30:28.39 +0800","os_version":"iPhone OS 13.3.1(17D50)","incident_id":"7F111601-BC7A-4BD7-A468-CE3370053057"}
{
  "crashReporterKey" :"bc2445adc164c399b330f812a48248e029e26276",
  "kernel" :"Darwin Kernel Version 19.3.0:Thu Jan  9 21:11:10 PST 2020; root:xnu-6153.82.3~1\/RELEASE_ARM64_T8030",
  "product" :"iPhone12,3",
  "incident" :"7F111601-BC7A-4BD7-A468-CE3370053057",
  "date" :"2020-03-19 17:30:28.39 +0800",
  "build" :"iPhone OS 13.3.1(17D50)",
  "timeDelta" :189,
  "memoryStatus" :{
  "compressorSize" :66443,
  "compressions" :25498129,
  "decompressions" :15532621,
  "zoneMapCap" :1395015680,
  "largestZone" :"APFS_4K_OBJS",
  "largestZoneSize" :41222144,
  "pageSize" :16384,
  "uncompressed" :127027,
  "zoneMapSize" :169639936,
  "memoryPages" :{
    "active" :58652,
    "throttled" :0,
    "fileBacked" :20291,
    "wired" :45838,
    "anonymous" :96445,
    "purgeable" :4,
    "inactive" :54368,
    "free" :5461,
    "speculative" :3716
  }
},
  "largestProcess" :"    ",
  "genCounter" :0,
  "processes" :[
  {
    "uuid" :"2dd5eb1e-fd31-36c2-99d9-bcbff44efbb7",
    "states" :[
      "daemon",
      "idle"
   ],
    "lifetimeMax" :171,
    "age" :5151034269954,
    "purgeable" :0,
    "fds" :50,
    "coalition" :66,
    "rpages" :164,
    "pid" :11276,
    "idleDelta" :3801132318,
    "name" :"wcd",
    "cpuTime" :3.430787
  },
  //...
  {
    "uuid" :"63158edc-915f-3a2b-975c-0e0ac4ed44c0",
    "states" :[
      "frontmost"
   ],
    "killDelta" :4345,
    "genCount" :0,
    "age" :654480778,
    "purgeable" :0,
    "fds" :50,
    "coalition" :1718,
    "rpages" :134278,
    "reason" :"per-process-limit",
    "pid" :14206,
    "cpuTime" :23.955463999999999,
    "name" :"    ",
    "lifetimeMax" :134278
  },
  //...
]
}

iPhone 11 Pro/13.3.1 OOM (16384*134278)/(1024*1024)=2098.09375M

*iOS Jetsam *

MacOS/iOS BSD Mach BSD Mach Mach BSD Jetsam BSD bsd_init

//1. Initialize the kernel memory allocator,     BSD    Zone    Zone     Mach    zone   
kmeminit();

//2. Initialise background freezing, iOS                       
#if CONFIG_FREEZE
#ifndef CONFIG_MEMORYSTATUS
    #error "CONFIG_FREEZE defined without matching CONFIG_MEMORYSTATUS"
#endif
    /* Initialise background freezing */
    bsd_init_kprintf("calling memorystatus_freeze_init\n");
    memorystatus_freeze_init();
#endif>

//3. iOS    JetSAM               
#if CONFIG_MEMORYSTATUS
    /* Initialize kernel memory status notifications */
    bsd_init_kprintf("calling memorystatus_init\n");
    memorystatus_init();
#endif /* CONFIG_MEMORYSTATUS */

** 2 **

CONFIG_FREEZE memorystatus_freeze_thread memorystatus_freeze_top_process

iOS vm_pressure_monitor App iOS Jetsam memorystatus XNU kern_memorystatus.h kern_memorystatus.c

iOS App 6 JetsamEvent 6

  iOS                **MemoryStatus        Jetsam**       iOS                 App                                               MacOS   MemoryStatus                 

MemoryStatus memorystatus_jetsam_thread App App

     App                     App      `didReceiveMemoryWarning`                                     App       

iOS

#define MEMSTAT_BUCKET_COUNT(JETSAM_PRIORITY_MAX + 1)

typedef struct memstat_bucket {
    TAILQ_HEAD(, proc) list;
    int count;
} memstat_bucket_t;

memstat_bucket_t memstat_bucket[MEMSTAT_BUCKET_COUNT];

kern_memorystatus.h

#define JETSAM_PRIORITY_IDLE_HEAD                -2
/* The value -1 is an alias to JETSAM_PRIORITY_DEFAULT */
#define JETSAM_PRIORITY_IDLE                      0
#define JETSAM_PRIORITY_IDLE_DEFERRED          1 /* Keeping this around till all xnu_quick_tests can be moved away from it.*/
#define JETSAM_PRIORITY_AGING_BAND1          JETSAM_PRIORITY_IDLE_DEFERRED
#define JETSAM_PRIORITY_BACKGROUND_OPPORTUNISTIC  2
#define JETSAM_PRIORITY_AGING_BAND2          JETSAM_PRIORITY_BACKGROUND_OPPORTUNISTIC
#define JETSAM_PRIORITY_BACKGROUND                3
#define JETSAM_PRIORITY_ELEVATED_INACTIVE      JETSAM_PRIORITY_BACKGROUND
#define JETSAM_PRIORITY_MAIL                      4
#define JETSAM_PRIORITY_PHONE                     5
#define JETSAM_PRIORITY_UI_SUPPORT                8
#define JETSAM_PRIORITY_FOREGROUND_SUPPORT        9
#define JETSAM_PRIORITY_FOREGROUND               10
#define JETSAM_PRIORITY_AUDIO_AND_ACCESSORY      12
#define JETSAM_PRIORITY_CONDUCTOR                13
#define JETSAM_PRIORITY_HOME                     16
#define JETSAM_PRIORITY_EXECUTIVE                17
#define JETSAM_PRIORITY_IMPORTANT                18
#define JETSAM_PRIORITY_CRITICAL                 19

#define JETSAM_PRIORITY_MAX                      21

       App     JETSAM\_PRIORITY\_BACKGROUND  3    App     JETSAM\_PRIORITY\_FOREGROUND  10 

           >         > App         App            App             CPU                

kern_memorystatus.c OOM

/* For logging clarity */
static const char *memorystatus_kill_cause_name[]= {
    ""                                ,        /* kMemorystatusInvalid                            */
    "jettisoned"                    ,        /* kMemorystatusKilled                            */
    "highwater"                        ,        /* kMemorystatusKilledHiwat                        */
    "vnode-limit"                    ,        /* kMemorystatusKilledVnodes                    */
    "vm-pageshortage"                ,        /* kMemorystatusKilledVMPageShortage            */
    "proc-thrashing"                ,        /* kMemorystatusKilledProcThrashing                */
    "fc-thrashing"                    ,        /* kMemorystatusKilledFCThrashing                */
    "per-process-limit"                ,        /* kMemorystatusKilledPerProcessLimit            */
    "disk-space-shortage"            ,        /* kMemorystatusKilledDiskSpaceShortage            */
    "idle-exit"                        ,        /* kMemorystatusKilledIdleExit                    */
    "zone-map-exhaustion"            ,        /* kMemorystatusKilledZoneMapExhaustion            */
    "vm-compressor-thrashing"        ,        /* kMemorystatusKilledVMCompressorThrashing        */
    "vm-compressor-space-shortage"    ,        /* kMemorystatusKilledVMCompressorSpaceShortage    */
};

memorystatus_init Jetsam

__private_extern__ void
memorystatus_init(void)
{
    //...
  /* Initialize the jetsam_threads state array */
    jetsam_threads = kalloc(sizeof(struct jetsam_thread_state) * max_jetsam_threads);

    /* Initialize all the jetsam threads */
    for(i = 0; i < max_jetsam_threads; i++) {

        result = kernel_thread_start_priority(memorystatus_thread, NULL, 95 /* MAXPRI_KERNEL */, &jetsam_threads[i].thread);
        if(result == KERN_SUCCESS) {
            jetsam_threads[i].inited = FALSE;
            jetsam_threads[i].index = i;
            thread_deallocate(jetsam_threads[i].thread);
        } else {
            panic("Could not create memorystatus_thread%d", i);
        }
    }
}

/*
 *    High-level priority assignments
 *
 *************************************************************************
 * 127        Reserved(real-time)
 *                A
 *                +
 *           (32 levels)
 *                +
 *                V
 * 96        Reserved(real-time)
 * 95        Kernel mode only
 *                A
 *                +
 *           (16 levels)
 *                +
 *                V
 * 80        Kernel mode only
 * 79        System high priority
 *                A
 *                +
 *           (16 levels)
 *                +
 *                V
 * 64        System high priority
 * 63        Elevated priorities
 *                A
 *                +
 *           (12 levels)
 *                +
 *                V
 * 52        Elevated priorities
 * 51        Elevated priorities(incl. BSD +nice)
 *                A
 *                +
 *           (20 levels)
 *                +
 *                V
 * 32        Elevated priorities(incl. BSD +nice)
 * 31        Default(default base for threads)
 * 30        Lowered priorities(incl. BSD -nice)
 *                A
 *                +
 *           (20 levels)
 *                +
 *                V
 * 11        Lowered priorities(incl. BSD -nice)
 * 10        Lowered priorities(aged pri's)
 *                A
 *                +
 *           (11 levels)
 *                +
 *                V
 * 0        Lowered priorities(aged pri's/idle)
 *************************************************************************
 */

                                                                               iOS              SpringBoard                 Mach                                CPU                                                            

                     max\_jetsam\_threads        1         3 jetsam               95     MAXPRI\_KERNEL       95         XNU           0 127                  -2~19  

    memorystatus\_thread                

static void
memorystatus_thread(void *param __unused, wait_result_t wr __unused)
{
  //...
  while(memorystatus_action_needed()) {
        boolean_t killed;
        int32_t priority;
        uint32_t cause;
        uint64_t jetsam_reason_code = JETSAM_REASON_INVALID;
        os_reason_t jetsam_reason = OS_REASON_NULL;

        cause = kill_under_pressure_cause;
        switch(cause) {
            case kMemorystatusKilledFCThrashing:
                jetsam_reason_code = JETSAM_REASON_MEMORY_FCTHRASHING;
                break;
            case kMemorystatusKilledVMCompressorThrashing:
                jetsam_reason_code = JETSAM_REASON_MEMORY_VMCOMPRESSOR_THRASHING;
                break;
            case kMemorystatusKilledVMCompressorSpaceShortage:
                jetsam_reason_code = JETSAM_REASON_MEMORY_VMCOMPRESSOR_SPACE_SHORTAGE;
                break;
            case kMemorystatusKilledZoneMapExhaustion:
                jetsam_reason_code = JETSAM_REASON_ZONE_MAP_EXHAUSTION;
                break;
            case kMemorystatusKilledVMPageShortage:
                /* falls through */
            default:
                jetsam_reason_code = JETSAM_REASON_MEMORY_VMPAGESHORTAGE;
                cause = kMemorystatusKilledVMPageShortage;
                break;
        }

        /* Highwater */
        boolean_t is_critical = TRUE;
        if(memorystatus_act_on_hiwat_processes(&errors, &hwm_kill, &post_snapshot, &is_critical)) {
            if(is_critical == FALSE) {
                /*
                 * For now, don't kill any other processes.
                 */
                break;
            } else {
                goto done;
            }
        }

        jetsam_reason = os_reason_create(OS_REASON_JETSAM, jetsam_reason_code);
        if(jetsam_reason == OS_REASON_NULL) {
            printf("memorystatus_thread:failed to allocate jetsam reason\n");
        }

        if(memorystatus_act_aggressive(cause, jetsam_reason, &jld_idle_kills, &corpse_list_purged, &post_snapshot)) {
            goto done;
        }

        /*
         * memorystatus_kill_top_process() drops a reference,
         * so take another one so we can continue to use this exit reason
         * even after it returns
         */
        os_reason_ref(jetsam_reason);

        /* LRU */
        killed = memorystatus_kill_top_process(TRUE, sort_flag, cause, jetsam_reason, &priority, &errors);
        sort_flag = FALSE;

        if(killed) {
            if(memorystatus_post_snapshot(priority, cause) == TRUE) {

                    post_snapshot = TRUE;
            }

            /* Jetsam Loop Detection */
            if(memorystatus_jld_enabled == TRUE) {
                if((priority == JETSAM_PRIORITY_IDLE) ||(priority == system_procs_aging_band) ||(priority == applications_aging_band)) {
                    jld_idle_kills++;
                } else {
                    /*
                     * We've reached into bands beyond idle deferred.
                     * We make no attempt to monitor them
                     */
                }
            }

            if((priority >= JETSAM_PRIORITY_UI_SUPPORT) &&(total_corpses_count() > 0) &&(corpse_list_purged == FALSE)) {
                /*
                 * If we have jetsammed a process in or above JETSAM_PRIORITY_UI_SUPPORT
                 * then we attempt to relieve pressure by purging corpse memory.
                 */
                task_purge_all_corpses();
                corpse_list_purged = TRUE;
            }
            goto done;
        }

        if(memorystatus_avail_pages_below_critical()) {
            /*
             * Still under pressure and unable to kill a process - purge corpse memory
             */
            if(total_corpses_count() > 0) {
                task_purge_all_corpses();
                corpse_list_purged = TRUE;
            }

            if(memorystatus_avail_pages_below_critical()) {
                /*
                 * Still under pressure and unable to kill a process - panic
                 */
                panic("memorystatus_jetsam_thread:no victim! available pages:%llu\n",(uint64_t)memorystatus_available_pages);
            }
        }

done:   

}

          memorystatus\_action\_needed()                

static boolean_t
memorystatus_action_needed(void)
{
#if CONFIG_EMBEDDED
    return(is_reason_thrashing(kill_under_pressure_cause) ||
            is_reason_zone_map_exhaustion(kill_under_pressure_cause) ||
           memorystatus_available_pages <= memorystatus_available_pages_pressure);
#else /* CONFIG_EMBEDDED */
    return(is_reason_thrashing(kill_under_pressure_cause) ||
            is_reason_zone_map_exhaustion(kill_under_pressure_cause));
#endif /* CONFIG_EMBEDDED */
}

vm\_pagepout                                     is\_reason\_thrashing, Mach Zone     is\_reason\_zone\_map\_exhaustion           memory status\_available\_pages      

memorystatus\_thread               High-water     OOM                              hight water mark      OOM   memorystatus\_act\_on\_hiwat\_processes()      memorystatus\_kill\_hiwat\_proc()        memstat\_bucket                         footprint\_in\_bytes <= memlimit\_in\_bytes                                   

   App       high water mark                 memorystatus\_act\_aggressive        OOM       

static boolean_t
memorystatus_act_aggressive(uint32_t cause, os_reason_t jetsam_reason, int *jld_idle_kills, boolean_t *corpse_list_purged, boolean_t *post_snapshot)
{
    //...
  if((jld_bucket_count == 0) || 
            (jld_now_msecs >(jld_timestamp_msecs + memorystatus_jld_eval_period_msecs))) {

            /* 
             * Refresh evaluation parameters 
             */
            jld_timestamp_msecs     = jld_now_msecs;
            jld_idle_kill_candidates = jld_bucket_count;
            *jld_idle_kills         = 0;
            jld_eval_aggressive_count = 0;
            jld_priority_band_max    = JETSAM_PRIORITY_UI_SUPPORT;
        }
  //...
}

             kill                  `jld_now_msecs >(jld_timestamp_msecs + memorystatus_jld_eval_period_msecs`       memorystatus\_jld\_eval\_period\_msecs           kill 

/* Jetsam Loop Detection */
if(max_mem <=(512 * 1024 * 1024)) {
    /* 512 MB devices */
memorystatus_jld_eval_period_msecs = 8000;    /* 8000 msecs == 8 second window */
} else {
    /* 1GB and larger devices */
memorystatus_jld_eval_period_msecs = 6000;    /* 6000 msecs == 6 second window */
}

memorystatus_jld_eval_period_msecs 6 6

3.2

stackoverflow OOM

device crash amount:MB total amount:MB percentage of total
iPad1 127 256 49%
iPad2 275 512 53%
iPad3 645 1024 62%
iPad4(iOS 8.1) 585 1024 57%
Pad Mini 1st Generation 297 512 58%
iPad Mini retina(iOS 7.1) 696 1024 68%
iPad Air 697 1024 68%
iPad Air 2(iOS 10.2.1) 1383 2048 68%
iPad Pro 9.7"(iOS 10.0.2(14A456)) 1395 1971 71%
iPad Pro 10.5(iOS 11 beta4) 3057 4000 76%
iPad Pro 12.9 (2015)(iOS 11.2.1) 3058 3999 76%
iPad 10.2(iOS 13.2.3) 1844 2998 62%
iPod touch 4th gen(iOS 6.1.1) 130 256 51%
iPod touch 5th gen 286 512 56%
iPhone4 325 512 63%
iPhone4s 286 512 56%
iPhone5 645 1024 62%
iPhone5s 646 1024 63%
iPhone6(iOS 8.x) 645 1024 62%
iPhone6 Plus(iOS 8.x) 645 1024 62%
iPhone6s(iOS 9.2) 1396 2048 68%
iPhone6s Plus(iOS 10.2.1) 1396 2048 68%
iPhoneSE(iOS 9.3) 1395 2048 68%
iPhone7(iOS 10.2) 1395 2048 68%
iPhone7 Plus(iOS 10.2.1) 2040 3072 66%
iPhone8(iOS 12.1) 1364 1990 70%
iPhoneX(iOS 11.2.1) 1392 2785 50%
iPhoneXS(iOS 12.1) 2040 3754 54%
iPhoneXS Max(iOS 12.1) 2039 3735 55%
iPhoneXR(iOS 12.1) 1792 2813 63%
iPhone11(iOS 13.1.3) 2068 3844 54%
iPhone11 Pro Max(iOS 13.2.3) 2067 3740 55%

3.3 App high water mark

                   `phys_footprint`                          Jetsam       App   **                      ** 

timer = [NSTimer scheduledTimerWithTimeInterval:0.01 target:self selector:@selector(allocateMemory) userInfo:nil repeats:YES];

-(void)allocateMemory {
    UIImageView *imageView = [[UIImageView alloc]initWithFrame:CGRectMake(0, 0, 100, 100)];
    UIImage *image = [UIImage imageNamed:@"AppIcon"];
    imageView.image = image;
    [array addObject:imageView];

    memoryLimitSizeMB = [self usedSizeOfMemory];
    if(memoryWarningSizeMB && memoryLimitSizeMB) {
        NSLog(@"----- memory warnning:%dMB, memory limit:%dMB", memoryWarningSizeMB, memoryLimitSizeMB);
    }
}

-(int)usedSizeOfMemory {
    task_vm_info_data_t taskInfo;
    mach_msg_type_number_t infoCount = TASK_VM_INFO_COUNT;
    kern_return_t kernReturn = task_info(mach_task_self(), TASK_VM_INFO,(task_info_t)&taskInfo, &infoCount);

    if(kernReturn != KERN_SUCCESS) {
        return 0;
    }
    return(int)(taskInfo.phys_footprint/1024.0/1024.0);
}

3.4 iOS13

iOS13 <os/proc.h> size_t os_proc_available_memory(void);

Return Value

The number of bytes that the app may allocate before it hits its memory limit. If the calling process isn't an app, or if the process has already exceeded its memory limit, this function returns 0.

Discussion

Call this function to determine the amount of memory available to your app. The returned value corresponds to the current memory limit minus the memory footprint of your app at the time of the function call. Your app's memory footprint consists of the data that you allocated in RAM, and that must stay in RAM(or the equivalent) at all times. Memory limits can change during the app life cycle and don't necessarily correspond to the amount of physical memory available on the device.

Use the returned value as advisory information only and don't cache it. The precise value changes when your app does any work that affects memory, which can happen frequently.

Although this function lets you determine the amount of memory your app may safely consume, don't use it to maximize your app's memory usage. Significant memory use, even when under the current memory limit, affects system performance. For example, when your app consumes all of its available memory, the system may need to terminate other apps and system processes to accommodate your app's requests. Instead, always consume the smallest amount of memory you need to be responsive to the user's needs.

If you need more detailed information about the available memory resources, you can call task_info. However, be aware that task_info is an expensive call, whereas this function is much more efficient.

if(@available(iOS 13.0, *)) {
    return os_proc_available_memory()/1024.0/1024.0;
}

App API Mach mach_task_basic_info Mach task phys_footprint virtual_size

#define MACH_TASK_BASIC_INFO     20         /* always 64-bit basic info */
struct mach_task_basic_info {
    mach_vm_size_t  virtual_size;       /* virtual memory size(bytes) */
    mach_vm_size_t  resident_size;      /* resident memory size(bytes) */
    mach_vm_size_t  resident_size_max;  /* maximum resident memory size(bytes) */
    time_value_t    user_time;          /* total user run time for
                                            terminated threads */
    time_value_t    system_time;        /* total system run time for
                                            terminated threads */
    policy_t        policy;             /* default policy for new threads */
    integer_t       suspend_count;      /* suspend count for task */
};



task_vm_info_data_t vmInfo;
mach_msg_type_number_t count = TASK_VM_INFO_COUNT;
kern_return_t kr = task_info(mach_task_self(), TASK_VM_INFO,(task_info_t)&vmInfo, &count);

if(kr != KERN_SUCCESS) {
    return;
}
CGFloat memoryUsed =(CGFloat)(vmInfo.phys_footprint/1024.0/1024.0);

       `resident_size`                         resident\_size   Xcode              phys\_footprint      Xcode            [WebKit  ](https://github.com/WebKit/webkit/blob/52bc6f0a96a062cb0eb76e9a81497183dc87c268/Source/WTF/wtf/cocoa/MemoryFootprintCocoa.cpp)      

iOS13          `os_proc_available_memory`               `phys_footprint`       App      2                      Jetsam    

-(CGFloat)limitSizeOfMemory {
    if(@available(iOS 13.0, *)) {
        task_vm_info_data_t taskInfo;
        mach_msg_type_number_t infoCount = TASK_VM_INFO_COUNT;
        kern_return_t kernReturn = task_info(mach_task_self(), TASK_VM_INFO,(task_info_t)&taskInfo, &infoCount);

        if(kernReturn != KERN_SUCCESS) {
            return 0;
        }
        return(CGFloat)((taskInfo.phys_footprint + os_proc_available_memory())/(1024.0 * 1024.0);
    }
    return 0;
}

     1435.936752MB    App       14.5MB     1435.936752MB + 14.5MB= 1450.436MB    3.1                iPhone 6s plus/13.3.1    OOM     (16384\*92806)/(1024\*1024)=1450.09375M  

3.5 XNU

XNU memorystatus_priority_entry

typedef struct memorystatus_priority_entry {
  pid_t pid;
  int32_t priority;
  uint64_t user_data;
  int32_t limit;
  uint32_t state;
} memorystatus_priority_entry_t;

priority limit root

    `kern_memorystatus.h`           `int memorystatus_control(uint32_t command, int32_t pid, uint32_t flags, void *buffer, size_t buffersize);`

/* Commands */
#define MEMORYSTATUS_CMD_GET_PRIORITY_LIST            1
#define MEMORYSTATUS_CMD_SET_PRIORITY_PROPERTIES      2
#define MEMORYSTATUS_CMD_GET_JETSAM_SNAPSHOT          3
#define MEMORYSTATUS_CMD_GET_PRESSURE_STATUS          4
#define MEMORYSTATUS_CMD_SET_JETSAM_HIGH_WATER_MARK   5    /* Set active memory limit = inactive memory limit, both non-fatal    */
#define MEMORYSTATUS_CMD_SET_JETSAM_TASK_LIMIT          6    /* Set active memory limit = inactive memory limit, both fatal    */
#define MEMORYSTATUS_CMD_SET_MEMLIMIT_PROPERTIES      7    /* Set memory limits plus attributes independently            */
#define MEMORYSTATUS_CMD_GET_MEMLIMIT_PROPERTIES      8    /* Get memory limits plus attributes                    */
#define MEMORYSTATUS_CMD_PRIVILEGED_LISTENER_ENABLE   9    /* Set the task's status as a privileged listener w.r.t memory notifications  */
#define MEMORYSTATUS_CMD_PRIVILEGED_LISTENER_DISABLE  10   /* Reset the task's status as a privileged listener w.r.t memory notifications  */
#define MEMORYSTATUS_CMD_AGGRESSIVE_JETSAM_LENIENT_MODE_ENABLE  11   /* Enable the 'lenient' mode for aggressive jetsam. See comments in kern_memorystatus.c near the top. */
#define MEMORYSTATUS_CMD_AGGRESSIVE_JETSAM_LENIENT_MODE_DISABLE 12   /* Disable the 'lenient' mode for aggressive jetsam. */
#define MEMORYSTATUS_CMD_GET_MEMLIMIT_EXCESS          13   /* Compute how much a process's phys_footprint exceeds inactive memory limit */
#define MEMORYSTATUS_CMD_ELEVATED_INACTIVEJETSAMPRIORITY_ENABLE     14 /* Set the inactive jetsam band for a process to JETSAM_PRIORITY_ELEVATED_INACTIVE */
#define MEMORYSTATUS_CMD_ELEVATED_INACTIVEJETSAMPRIORITY_DISABLE     15 /* Reset the inactive jetsam band for a process to the default band(0)*/
#define MEMORYSTATUS_CMD_SET_PROCESS_IS_MANAGED       16   /*(Re-)Set state on a process that marks it as(un-)managed by a system entity e.g. assertiond */
#define MEMORYSTATUS_CMD_GET_PROCESS_IS_MANAGED       17   /* Return the 'managed' status of a process */
#define MEMORYSTATUS_CMD_SET_PROCESS_IS_FREEZABLE     18   /* Is the process eligible for freezing? Apps and extensions can pass in FALSE to opt out of freezing, i.e.,



struct memorystatus_priority_entry memStatus[NUM_ENTRIES];
size_t count = sizeof(struct memorystatus_priority_entry) * NUM_ENTRIES;
int kernResult = memorystatus_control(MEMORYSTATUS_CMD_GET_PRIORITY_LIST, 0, 0, memStatus, count);
if(rc < 0) {
  NSLog(@"memorystatus_control"); 
    return;
}

int entry = 0;
for(; rc > 0; rc -= sizeof(struct memorystatus_priority_entry)){
  printf("PID:%5d\tPriority:%2d\tUser Data:%llx\tLimit:%2d\tState:%s\n",
          memstatus[entry].pid,
          memstatus[entry].priority,
          memstatus[entry].user_data,
          memstatus[entry].limit,
          state_to_text(memstatus[entry].state));
  entry++;
}

for App pid Priority User Data Limit State log 10 App 10 #define JETSAM_PRIORITY_FOREGROUND 10 App

4. OOM

OOM crash app

2

// 1
NSMutableArray *array = [NSMutableArray array];
for(NSInteger index = 0; index < 10000000; index++) {
  NSString *filePath = [[NSBundle mainBundle]pathForResource:@"Info" ofType:@"plist"];
  NSData *data = [NSData dataWithContentsOfFile:filePath];
  [array addObject:data];
}

// 2
//ViewController.m
-(void)viewDidLoad {
    [super viewDidLoad];
    dispatch_async(dispatch_get_global_queue(0, 0), ^{
        NSMutableArray *array = [NSMutableArray array];
        for(NSInteger index = 0; index < 10000000; index++) {
            NSString *filePath = [[NSBundle mainBundle]pathForResource:@"Info" ofType:@"plist"];
            NSData *data = [NSData dataWithContentsOfFile:filePath];
            [array addObject:data];
        }
    });
}
-(void)didReceiveMemoryWarning
{
    NSLog(@"2");
}

//AppDelegate.m
-(void)applicationDidReceiveMemoryWarning:(UIApplication *)application
{
    NSLog(@"1");
}
  1. viewDidLoad Crash
  2. App AppDelegate applicationDidReceiveMemoryWarning VC didReceiveMemoryWarning

** Crash 6 6 crash OOM **

5.

          dump                                                              



libmalloc/malloc         malloc   calloc       nano\_zone nano\_zone     256B            256B     scalable\_zone     

         malloc       malloc\_zone\_malloc, calloc     malloc\_zone\_calloc 

scalable_zone malloc_logger

void *
malloc(size_t size)
{
    void *retval;
    retval = malloc_zone_malloc(default_zone, size);
    if(retval == NULL) {
        errno = ENOMEM;
    }
    return retval;
}

void *
calloc(size_t num_items, size_t size)
{
    void *retval;
    retval = malloc_zone_calloc(default_zone, num_items, size);
    if(retval == NULL) {
        errno = ENOMEM;
    }
    return retval;
}

    `default_zone`      ,     

typedef struct {
    malloc_zone_t malloc_zone;
    uint8_t pad[PAGE_MAX_SIZE - sizeof(malloc_zone_t)];
} virtual_default_zone_t;

static virtual_default_zone_t virtual_default_zone
__attribute__((section("__DATA,__v_zone")))
__attribute__((aligned(PAGE_MAX_SIZE))) = {
    NULL,
    NULL,
    default_zone_size,
    default_zone_malloc,
    default_zone_calloc,
    default_zone_valloc,
    default_zone_free,
    default_zone_realloc,
    default_zone_destroy,
    DEFAULT_MALLOC_ZONE_STRING,
    default_zone_batch_malloc,
    default_zone_batch_free,
    &default_zone_introspect,
    10,
    default_zone_memalign,
    default_zone_free_definite_size,
    default_zone_pressure_relief,
    default_zone_malloc_claimed_address,
};

static malloc_zone_t *default_zone = &virtual_default_zone.malloc_zone;

static void *
default_zone_malloc(malloc_zone_t *zone, size_t size)
{
    zone = runtime_default_zone();

    return zone->malloc(zone, size);
}


MALLOC_ALWAYS_INLINE
static inline malloc_zone_t *
runtime_default_zone() {
    return(lite_zone) ? lite_zone :inline_malloc_default_zone();
}

 `default_zone`           

static inline malloc_zone_t *
inline_malloc_default_zone(void)
{
    _malloc_initialize_once();
    //malloc_report(ASL_LEVEL_INFO, "In inline_malloc_default_zone with%d%d\n", malloc_num_zones, malloc_has_debug_zone);
    return malloc_zones[0];
}

_malloc_initialize -> create_scalable_zone -> create_scalable_szone szone_t default_zone

malloc_zone_t *
create_scalable_zone(size_t initial_size, unsigned debug_flags) {
    return(malloc_zone_t *) create_scalable_szone(initial_size, debug_flags);
}

void *malloc_zone_malloc(malloc_zone_t *zone, size_t size)
{
  MALLOC_TRACE(TRACE_malloc | DBG_FUNC_START,(uintptr_t)zone, size, 0, 0);
  void *ptr;
  if(malloc_check_start &&(malloc_check_counter++ >= malloc_check_start)) {
    internal_check();
  }
  if(size > MALLOC_ABSOLUTE_MAX_SIZE) {
    return NULL;
  }
  ptr = zone->malloc(zone, size);
  // zone             malloc_logger       
  if(malloc_logger) {
    malloc_logger(MALLOC_LOG_TYPE_ALLOCATE | MALLOC_LOG_TYPE_HAS_ZONE,(uintptr_t)zone,(uintptr_t)size, 0,(uintptr_t)ptr, 0);
  }
  MALLOC_TRACE(TRACE_malloc | DBG_FUNC_END,(uintptr_t)zone, size,(uintptr_t)ptr, 0);
  return ptr;
}

   `zone->malloc`           szone\_t         malloc   

szone

//Initialize the security token.
szone->cookie =(uintptr_t)malloc_entropy[0];

szone->basic_zone.version = 12;
szone->basic_zone.size =(void *)szone_size;
szone->basic_zone.malloc =(void *)szone_malloc;
szone->basic_zone.calloc =(void *)szone_calloc;
szone->basic_zone.valloc =(void *)szone_valloc;
szone->basic_zone.free =(void *)szone_free;
szone->basic_zone.realloc =(void *)szone_realloc;
szone->basic_zone.destroy =(void *)szone_destroy;
szone->basic_zone.batch_malloc =(void *)szone_batch_malloc;
szone->basic_zone.batch_free =(void *)szone_batch_free;
szone->basic_zone.introspect =(struct malloc_introspection_t *)&szone_introspect;
szone->basic_zone.memalign =(void *)szone_memalign;
szone->basic_zone.free_definite_size =(void *)szone_free_definite_size;
szone->basic_zone.pressure_relief =(void *)szone_pressure_relief;
szone->basic_zone.claimed_address =(void *)szone_claimed_address;

 scalable\_zone                                           malloc\_logger            fishhook   hook                                          

//For logging VM allocation and deallocation, arg1 here
//is the mach_port_name_t of the target task in which the
//alloc or dealloc is occurring. For example, for mmap()
//that would be mach_task_self(), but for a cross-task-capable
//call such as mach_vm_map(), it is the target task.

typedef void(malloc_logger_t)(uint32_t type, uintptr_t arg1, uintptr_t arg2, uintptr_t arg3, uintptr_t result, uint32_t num_hot_frames_to_skip);

extern malloc_logger_t *__syscall_logger;

malloc_logger __syscall_logger malloc/free vm_allocate/vm_deallocate / malloc stack backtrace dsym image slide ** = - slide **

tips

ASLR Address space layout randomization

 add:         ;

   `vm_add`;

ASLR:slide mach-o slide vm_add + slide = add *(base +offset)= imp

        OOM     - [OOMDetector](https://github.com/Tencent/OOMDetector)                                      App                     dump                                                                                                    owner                                       
              hot fix 

6.

  1. WWDC 2018 Session 416 - iOS Memory Deep Dive UIImage bitmap ImageIO 2

    2 App

    carbon.png

    ImageIO UIImage

  2. autoreleasepool

    autoreleasepool RunLoop ARC autoreleasepool OOM

![carbon(1).png](https://i0.wp.com/segmentfault.com/img/bVbINDv "carbon(1).png")

  1     739.6M   2     587M 
  1. UIGraphicsBeginImageContext UIGraphicsEndImageContext context XCode Analyze
  2. js WKWebView UIWebView App OOM WKWebView Network Loading UI Rendering UIWebView
  3. SDK App NSCache NSMutableDictionary NSCache Purgeable Memory NSCache NSPureableData

App

        WIFI 2G 3G 4G 5G        App                                         Connection Migration        DNS                          App                                       

1. App

App

  • DNS

    Domain Name system IP DNS DNS ** ** App DNS App DNS App HTTPDNS DNS

  • TCP 3

    TCP 3 2 4

  • TLS

    HTTPS TLS

  •      request         request start   
  •        HTTP header                                 

2.

NSURLConnection
NSURLSession iOS7.0
CFNetwork NSURL C

iOS
Network Level

iOS 4 BSD Sockets SecureTransport CFNetwork NSURLSession NSURLConnection WebView Objective-C CFNetwork AFNetworking NSURLSession NSURLConnection

         2        NSURLProtocol          Hook                           

2.1 NSURLProtocol App

NSURLProtocol NSURLProtocol URL Loading System FTP HTTP HTTPS CFNetwork

NSURLProtocol

iOS 10 NSURLSessionTaskDelegate

/*
 * Sent when complete statistics information has been collected for the task.
 */
-(void)URLSession:(NSURLSession *)session task:(NSURLSessionTask *)task didFinishCollectingMetrics:(NSURLSessionTaskMetrics *)metrics API_AVAILABLE(macosx(10.12), ios(10.0), watchos(3.0), tvos(10.0));

`NSURLSessionTaskMetrics`                     

@interface NSURLSessionTaskMetrics :NSObject

/*
 * transactionMetrics array contains the metrics collected for every request/response transaction created during the task execution.
 */
@property(copy, readonly) NSArray<NSURLSessionTaskTransactionMetrics *> *transactionMetrics;

/*
 * Interval from the task creation time to the task completion time.
 * Task creation time is the time when the task was instantiated.
 * Task completion time is the time when the task is about to change its internal state to completed.
 */
@property(copy, readonly) NSDateInterval *taskInterval;

/*
 * redirectCount is the number of redirects that were recorded.
 */
@property(assign, readonly) NSUInteger redirectCount;

-(instancetype)init API_DEPRECATED("Not supported", macos(10.12,10.15), ios(10.0,13.0), watchos(3.0,6.0), tvos(10.0,13.0));
+(instancetype)new API_DEPRECATED("Not supported", macos(10.12,10.15), ios(10.0,13.0), watchos(3.0,6.0), tvos(10.0,13.0));

@end

taskInterval redirectCount transactionMetrics /

/*
 * This class defines the performance metrics collected for a request/response transaction during the task execution.
 */
API_AVAILABLE(macosx(10.12), ios(10.0), watchos(3.0), tvos(10.0))
@interface NSURLSessionTaskTransactionMetrics :NSObject

/*
 * Represents the transaction request.     
 */
@property(copy, readonly) NSURLRequest *request;

/*
 * Represents the transaction response. Can be nil if error occurred and no response was generated.     
 */
@property(nullable, copy, readonly) NSURLResponse *response;

/*
 * For all NSDate metrics below, if that aspect of the task could not be completed, then the corresponding  EndDate  metric will be nil.
 * For example, if a name lookup was started but the name lookup timed out, failed, or the client canceled the task before the name could be resolved -- then while domainLookupStartDate may be set, domainLookupEndDate will be nil along with all later metrics.
 */

/*
 *                             
 * fetchStartDate returns the time when the user agent started fetching the resource, whether or not the resource was retrieved from the server or local resources.
 *
 * The following metrics will be set to nil, if a persistent connection was used or the resource was retrieved from local resources:
 *
 *   domainLookupStartDate
 *   domainLookupEndDate
 *   connectStartDate
 *   connectEndDate
 *   secureConnectionStartDate
 *   secureConnectionEndDate
 */
@property(nullable, copy, readonly) NSDate *fetchStartDate;

/*
 * domainLookupStartDate returns the time immediately before the user agent started the name lookup for the resource. DNS        
 */
@property(nullable, copy, readonly) NSDate *domainLookupStartDate;

/*
 * domainLookupEndDate returns the time after the name lookup was completed. DNS        
 */
@property(nullable, copy, readonly) NSDate *domainLookupEndDate;

/*
 * connectStartDate is the time immediately before the user agent started establishing the connection to the server.
 *
 * For example, this would correspond to the time immediately before the user agent started trying to establish the TCP connection.             TCP      
 */
@property(nullable, copy, readonly) NSDate *connectStartDate;

/*
 * If an encrypted connection was used, secureConnectionStartDate is the time immediately before the user agent started the security handshake to secure the current connection. HTTPS   TLS        
 *
 * For example, this would correspond to the time immediately before the user agent started the TLS handshake. 
 *
 * If an encrypted connection was not used, this attribute is set to nil.
 */
@property(nullable, copy, readonly) NSDate *secureConnectionStartDate;

/*
 * If an encrypted connection was used, secureConnectionEndDate is the time immediately after the security handshake completed. HTTPS   TLS        
 *
 * If an encrypted connection was not used, this attribute is set to nil.
 */
@property(nullable, copy, readonly) NSDate *secureConnectionEndDate;

/*
 * connectEndDate is the time immediately after the user agent finished establishing the connection to the server, including completion of security-related and other handshakes.           TCP            TLS     
 */
@property(nullable, copy, readonly) NSDate *connectEndDate;

/*
 * requestStartDate is the time immediately before the user agent started requesting the source, regardless of whether the resource was retrieved from the server or local resources.
                      HTTP     header         
 *
 * For example, this would correspond to the time immediately before the user agent sent an HTTP GET request.
 */
@property(nullable, copy, readonly) NSDate *requestStartDate;

/*
 * requestEndDate is the time immediately after the user agent finished requesting the source, regardless of whether the resource was retrieved from the server or local resources.
                  HTTP                 
 *
 * For example, this would correspond to the time immediately after the user agent finished sending the last byte of the request.
 */
@property(nullable, copy, readonly) NSDate *requestEndDate;

/*
 * responseStartDate is the time immediately after the user agent received the first byte of the response from the server or from local resources.

 *
 * For example, this would correspond to the time immediately after the user agent received the first byte of an HTTP response.
 */
@property(nullable, copy, readonly) NSDate *responseStartDate;

/*
 * responseEndDate is the time immediately after the user agent received the last byte of the resource.                    
 */
@property(nullable, copy, readonly) NSDate *responseEndDate;

/*
 * The network protocol used to fetch the resource, as identified by the ALPN Protocol ID Identification Sequence [RFC7301].
 * E.g., h2, http/1.1, spdy/3.1.
          http/1.1, spdy/3.1
 *
 * When a proxy is configured AND a tunnel connection is established, then this attribute returns the value for the tunneled protocol.
 *
 * For example:
 * If no proxy were used, and HTTP/2 was negotiated, then h2 would be returned.
 * If HTTP/1.1 were used to the proxy, and the tunneled connection was HTTP/2, then h2 would be returned.
 * If HTTP/1.1 were used to the proxy, and there were no tunnel, then http/1.1 would be returned.
 *
 */
@property(nullable, copy, readonly) NSString *networkProtocolName;

/*
 * This property is set to YES if a proxy connection was used to fetch the resource.

 */
@property(assign, readonly, getter=isProxyConnection) BOOL proxyConnection;

/*
 * This property is set to YES if a persistent connection was used to fetch the resource.

 */
@property(assign, readonly, getter=isReusedConnection) BOOL reusedConnection;

/*
 * Indicates whether the resource was loaded, pushed or retrieved from the local cache.

 */
@property(assign, readonly) NSURLSessionTaskMetricsResourceFetchType resourceFetchType;

/*
 * countOfRequestHeaderBytesSent is the number of bytes transferred for request header.

 */
@property(readonly) int64_t countOfRequestHeaderBytesSent API_AVAILABLE(macos(10.15), ios(13.0), watchos(6.0), tvos(13.0));

/*
 * countOfRequestBodyBytesSent is the number of bytes transferred for request body.

 * It includes protocol-specific framing, transfer encoding, and content encoding.
 */
@property(readonly) int64_t countOfRequestBodyBytesSent API_AVAILABLE(macos(10.15), ios(13.0), watchos(6.0), tvos(13.0));

/*
 * countOfRequestBodyBytesBeforeEncoding is the size of upload body data, file, or stream.

 */
@property(readonly) int64_t countOfRequestBodyBytesBeforeEncoding API_AVAILABLE(macos(10.15), ios(13.0), watchos(6.0), tvos(13.0));

/*
 * countOfResponseHeaderBytesReceived is the number of bytes transferred for response header.

 */
@property(readonly) int64_t countOfResponseHeaderBytesReceived API_AVAILABLE(macos(10.15), ios(13.0), watchos(6.0), tvos(13.0));

/*
 * countOfResponseBodyBytesReceived is the number of bytes transferred for response body.

 * It includes protocol-specific framing, transfer encoding, and content encoding.
 */
@property(readonly) int64_t countOfResponseBodyBytesReceived API_AVAILABLE(macos(10.15), ios(13.0), watchos(6.0), tvos(13.0));

/*
 * countOfResponseBodyBytesAfterDecoding is the size of data delivered to your delegate or completion handler.


 */
@property(readonly) int64_t countOfResponseBodyBytesAfterDecoding API_AVAILABLE(macos(10.15), ios(13.0), watchos(6.0), tvos(13.0));

/*
 * localAddress is the IP address string of the local interface for the connection.
             IP   
 *
 * For multipath protocols, this is the local address of the initial flow.
 *
 * If a connection was not used, this attribute is set to nil.
 */
@property(nullable, copy, readonly) NSString *localAddress API_AVAILABLE(macos(10.15), ios(13.0), watchos(6.0), tvos(13.0));

/*
 * localPort is the port number of the local interface for the connection.


 *
 * For multipath protocols, this is the local port of the initial flow.
 *
 * If a connection was not used, this attribute is set to nil.
 */
@property(nullable, copy, readonly) NSNumber *localPort API_AVAILABLE(macos(10.15), ios(13.0), watchos(6.0), tvos(13.0));

/*
 * remoteAddress is the IP address string of the remote interface for the connection.
          IP   
 *
 * For multipath protocols, this is the remote address of the initial flow.
 *
 * If a connection was not used, this attribute is set to nil.
 */
@property(nullable, copy, readonly) NSString *remoteAddress API_AVAILABLE(macos(10.15), ios(13.0), watchos(6.0), tvos(13.0));

/*
 * remotePort is the port number of the remote interface for the connection.

 *
 * For multipath protocols, this is the remote port of the initial flow.
 *
 * If a connection was not used, this attribute is set to nil.
 */
@property(nullable, copy, readonly) NSNumber *remotePort API_AVAILABLE(macos(10.15), ios(13.0), watchos(6.0), tvos(13.0));

/*
 * negotiatedTLSProtocolVersion is the TLS protocol version negotiated for the connection.
         TLS      
 * It is a 2-byte sequence in host byte order.
 *
 * Please refer to tls_protocol_version_t enum in Security/SecProtocolTypes.h
 *
 * If an encrypted connection was not used, this attribute is set to nil.
 */
@property(nullable, copy, readonly) NSNumber *negotiatedTLSProtocolVersion API_AVAILABLE(macos(10.15), ios(13.0), watchos(6.0), tvos(13.0));

/*
 * negotiatedTLSCipherSuite is the TLS cipher suite negotiated for the connection.
        TLS     
 * It is a 2-byte sequence in host byte order.
 *
 * Please refer to tls_ciphersuite_t enum in Security/SecProtocolTypes.h
 *
 * If an encrypted connection was not used, this attribute is set to nil.
 */
@property(nullable, copy, readonly) NSNumber *negotiatedTLSCipherSuite API_AVAILABLE(macos(10.15), ios(13.0), watchos(6.0), tvos(13.0));

/*
 * Whether the connection is established over a cellular interface.

 */
@property(readonly, getter=isCellular) BOOL cellular API_AVAILABLE(macos(10.15), ios(13.0), watchos(6.0), tvos(13.0));

/*
 * Whether the connection is established over an expensive interface.

 */
@property(readonly, getter=isExpensive) BOOL expensive API_AVAILABLE(macos(10.15), ios(13.0), watchos(6.0), tvos(13.0));

/*
 * Whether the connection is established over a constrained interface.

 */
@property(readonly, getter=isConstrained) BOOL constrained API_AVAILABLE(macos(10.15), ios(13.0), watchos(6.0), tvos(13.0));

/*
 * Whether a multipath protocol is successfully negotiated for the connection.

 */
@property(readonly, getter=isMultipath) BOOL multipath API_AVAILABLE(macos(10.15), ios(13.0), watchos(6.0), tvos(13.0));


-(instancetype)init API_DEPRECATED("Not supported", macos(10.12,10.15), ios(10.0,13.0), watchos(3.0,6.0), tvos(10.0,13.0));
+(instancetype)new API_DEPRECATED("Not supported", macos(10.12,10.15), ios(10.0,13.0), watchos(3.0,6.0), tvos(10.0,13.0));

@end



//     
@interface  NetworkMonitorBaseDataModel :NSObject
//   URL   
@property(nonatomic, strong) NSString *requestUrl;
// 
@property(nonatomic, strong) NSArray *requestHeaders;
// 
@property(nonatomic, strong) NSArray *responseHeaders;
//GET        
@property(nonatomic, strong) NSString *getRequestParams;
//HTTP   ,    POST
@property(nonatomic, strong) NSString *httpMethod;
//   http1.0/http1.1/http2.0
@property(nonatomic, strong) NSString *httpProtocol;
//    
@property(nonatomic, assign) BOOL useProxy;
//DNS     IP   
@property(nonatomic, strong) NSString *ip;
@end

//     
@interface  NetworkMonitorDataModel :NetworkMonitorBaseDataModel
//        
@property(nonatomic, assign) UInt64 requestDate;
//        dns       ,  ms 
@property(nonatomic, assign) int waitDNSTime;
//DNS     
@property(nonatomic, assign) int dnsLookupTime;
//tcp       ,  ms
@property(nonatomic, assign) int tcpTime;
//ssl     
@property(nonatomic, assign) int sslTime;
//       ,  ms
@property(nonatomic, assign) int requestTime;
//http    
@property(nonatomic, assign) NSUInteger httpCode;
//    
@property(nonatomic, assign) UInt64 sendBytes;
//    
@property(nonatomic, assign) UInt64 receiveBytes;


//     
@interface  NetworkMonitorErrorModel :NetworkMonitorBaseDataModel
// 
@property(nonatomic, assign) NSInteger errorCode;
//  
@property(nonatomic, assign) NSUInteger errCount;
// 
@property(nonatomic, strong) NSString *exceptionName;
//  
@property(nonatomic, strong) NSString *exceptionDetail;
//  
@property(nonatomic, strong) NSString *stackTrace;
@end


//   NSURLProtocol                  
@interface CustomURLProtocol() <NSURLSessionTaskDelegate>

@property(nonatomic, strong) NSURLSessionDataTask *dataTask;
@property(nonatomic, strong) NSOperationQueue *sessionDelegateQueue;
@property(nonatomic, strong) NetworkMonitorDataModel *dataModel;
@property(nonatomic, strong) NetworkMonitorErrorModel *errModel;

@end

//NSURLSessionDataTask    
-(void)startLoading {
    NSURLSessionConfiguration *configuration = [NSURLSessionConfiguration defaultSessionConfiguration];
      NSURLSession *session = [NSURLSession sessionWithConfiguration:configuration
                                                          delegate:self
                                                     delegateQueue:nil];
    NSURLSession *session = [NSURLSession sessionWithConfiguration:configuration delegate:self delegateQueue:nil];
      self.sessionDelegateQueue = [[NSOperationQueue alloc]init];
    self.sessionDelegateQueue.maxConcurrentOperationCount = 1;
    self.sessionDelegateQueue.name = @"com.networkMonitor.session.queue";
    self.dataTask = [session dataTaskWithRequest:self.request];
    [self.dataTask resume];
}

#pragma mark - NSURLSessionTaskDelegate
-(void)URLSession:(NSURLSession *)session task:(NSURLSessionTask *)task didCompleteWithError:(NSError *)error {
    if(error) {
        [self.client URLProtocol:self didFailWithError:error];
    } else {
        [self.client URLProtocolDidFinishLoading:self];
    }
    if(error) {
        NSURLRequest *request = task.currentRequest;
        if(request) {
            self.errModel.requestUrl  = request.URL.absoluteString;        
            self.errModel.httpMethod = request.HTTPMethod;
            self.errModel.requestParams = request.URL.query;
        }
        self.errModel.errorCode = error.code;
        self.errModel.exceptionName = error.domain;
        self.errModel.exceptionDetail = error.description;
      //  Network                  [                 ](https://github.com/FantasticLBP/knowledge-kit/blob/master/Chapter1%20-%20iOS/1.80.md)  
    }
    self.dataTask = nil;
}


-(void)URLSession:(NSURLSession *)session task:(NSURLSessionTask *)task didFinishCollectingMetrics:(NSURLSessionTaskMetrics *)metrics {
       if(@available(iOS 10.0, *) && [metrics.transactionMetrics count]> 0) {
        [metrics.transactionMetrics enumerateObjectsUsingBlock:^(NSURLSessionTaskTransactionMetrics *_Nonnull obj, NSUInteger idx, BOOL *_Nonnull stop) {
            if(obj.resourceFetchType == NSURLSessionTaskMetricsResourceFetchTypeNetworkLoad) {
                if(obj.fetchStartDate) {
                    self.dataModel.requestDate = [obj.fetchStartDate timeIntervalSince1970]* 1000;
                }
                if(obj.domainLookupStartDate && obj.domainLookupEndDate) {
                    self.dataModel. waitDNSTime = ceil([obj.domainLookupStartDate timeIntervalSinceDate:obj.fetchStartDate]* 1000);
                    self.dataModel. dnsLookupTime = ceil([obj.domainLookupEndDate timeIntervalSinceDate:obj.domainLookupStartDate]* 1000);
                }
                if(obj.connectStartDate) {
                    if(obj.secureConnectionStartDate) {
                        self.dataModel. waitDNSTime = ceil([obj.secureConnectionStartDate timeIntervalSinceDate:obj.connectStartDate]* 1000);
                    } else if(obj.connectEndDate) {
                        self.dataModel.tcpTime = ceil([obj.connectEndDate timeIntervalSinceDate:obj.connectStartDate]* 1000);
                    }
                }
                if(obj.secureConnectionEndDate && obj.secureConnectionStartDate) {
                    self.dataModel.sslTime = ceil([obj.secureConnectionEndDate timeIntervalSinceDate:obj.secureConnectionStartDate]* 1000);
                }

                if(obj.fetchStartDate && obj.responseEndDate) {
                    self.dataModel.requestTime = ceil([obj.responseEndDate timeIntervalSinceDate:obj.fetchStartDate]* 1000);
                }

                self.dataModel.httpProtocol = obj.networkProtocolName;

                NSHTTPURLResponse *response =(NSHTTPURLResponse *)obj.response;
                if([response isKindOfClass:NSHTTPURLResponse.class]) {
                    self.dataModel.receiveBytes = response.expectedContentLength;
                }

                if([obj respondsToSelector:@selector(_remoteAddressAndPort)]) {
                    self.dataModel.ip = [obj valueForKey:@"_remoteAddressAndPort"];
                }

                if([obj respondsToSelector:@selector(_requestHeaderBytesSent)]) {
                    self.dataModel.sendBytes = [[obj valueForKey:@"_requestHeaderBytesSent"]unsignedIntegerValue];
                }
                if([obj respondsToSelector:@selector(_responseHeaderBytesReceived)]) {
                    self.dataModel.receiveBytes = [[obj valueForKey:@"_responseHeaderBytesReceived"]unsignedIntegerValue];
                }

               self.dataModel.requestUrl = [obj.request.URL absoluteString];
                self.dataModel.httpMethod = obj.request.HTTPMethod;
                self.dataModel.useProxy = obj.isProxyConnection;
            }
        }];
                //  Network                  [                 ](https://github.com/FantasticLBP/knowledge-kit/blob/master/Chapter1%20-%20iOS/1.80.md)  
    }
}

2.2 NSURLProtocol App

 [2.1](#network-2.1)      NSURLSessionTaskMetrics                                        [](https://www.jianshu.com/p/1c34147030d1)       WebView            Webkit             

#if !HAVE(TIMINGDATAOPTIONS)
void setCollectsTimingData()
{
    static dispatch_once_t onceToken;
    dispatch_once(&onceToken, ^{
        [NSURLConnection _setCollectsTimingData:YES];
        ...
    });
}
#endif

  NSURLConnection       `TimingData`     API                       runtime header      NSURLConnection   `_setCollectsTimingData:`  `_timingData` 2  api iOS8         

NSURLSession iOS9 _setCollectsTimingData: TimingData

note:

  • API [[@"_setC" stringByAppendingString:@"ollectsT"]stringByAppendingString:@"imingData:"]

  • API APM SDK App

    @interface _NSURLConnectionProxy :DelegateProxy

    @end

    @implementation _NSURLConnectionProxy

    -(BOOL)respondsToSelector:(SEL)aSelector
    {
    if([NSStringFromSelector(aSelector) isEqualToString:@"connectionDidFinishLoading:"]) {
    return YES;
    }
    return [self.target respondsToSelector:aSelector];
    }

    -(void)forwardInvocation:(NSInvocation *)invocation
    {
    [super forwardInvocation:invocation];
    if([NSStringFromSelector(invocation.selector) isEqualToString:@"connectionDidFinishLoading:"]) {
    __unsafe_unretained NSURLConnection *conn;
    [invocation getArgument:&conn atIndex:2];
    SEL selector = NSSelectorFromString([@"_timin" stringByAppendingString:@"gData"]);
    NSDictionary *timingData = [conn performSelector:selector];
    [[NTDataKeeper shareInstance]trackTimingData:timingData request:conn.currentRequest];
    }
    }

    @end

    @implementation NSURLConnection(tracker)

    +(void)load
    {
    static dispatch_once_t onceToken;
    dispatch_once(&onceToken, ^{
    Class class = [self class];

    SEL originalSelector = @selector(initWithRequest:delegate:);
    SEL swizzledSelector = @selector(swizzledInitWithRequest:delegate:);

    Method originalMethod = class_getInstanceMethod(class, originalSelector);
    Method swizzledMethod = class_getInstanceMethod(class, swizzledSelector);
    method_exchangeImplementations(originalMethod, swizzledMethod);

    NSString *selectorName = [[@"_setC" stringByAppendingString:@"ollectsT"]stringByAppendingString:@"imingData:"];
    SEL selector = NSSelectorFromString(selectorName);
    [NSURLConnection performSelector:selector withObject:@(YES)];
    });
    }

    -(instancetype)swizzledInitWithRequest:(NSURLRequest *)request delegate:(id)delegate
    {
    if(delegate) {
    _NSURLConnectionProxy *proxy = [[_NSURLConnectionProxy alloc]initWithTarget:delegate];
    objc_setAssociatedObject(delegate ,@"_NSURLConnectionProxy" ,proxy, OBJC_ASSOCIATION_RETAIN_NONATOMIC);
    return [self swizzledInitWithRequest:request delegate:(id)proxy];
    }else{
    return [self swizzledInitWithRequest:request delegate:delegate];
    }
    }

    @end

2.3 Hook

iOS hook 2 NSProxy method swizzling isa swizzling

2.3.1

SDK ? APM Hook

     Aspect-oriented Programming AOP                   **   **                                                                                                                                                          

iOS AOP Runtime 3 Method Swizzling NSProxy FishHook hook c

 [2.1](#network-2.1)                NSURLProtocol     NSURLConnection NSURLSession                                       header                          DNS        DNS        reponse                 iOS10    NSURLSessionTaskDelegate           `-(void)URLSession:(NSURLSession *)session task:(NSURLSessionTask *)task didFinishCollectingMetrics:(NSURLSessionTaskMetrics *)metrics API_AVAILABLE(macosx(10.12), ios(10.0), watchos(3.0), tvos(10.0));`                             [2.2](#network-2.2)      Webkit                 `_setCollectsTimingData:`  `_timingData`       TimingData 

                                 APM             3                 

network hook

 CFNetwork            CFNetwork           

CFNetwork Structure

CFNetwork CFSocket CFStream

CFSocket Socket 2 socket iOS socket BSD socket CFSocket BSD socket OC BSD RunLoop

CFStream socket stream CFStream API 2 CFType CFReadStream CFWriteStream CFHTTP CFFTP

Demo

-(void)testCFNetwork
{
    CFURLRef urlRef = CFURLCreateWithString(kCFAllocatorDefault, CFSTR("https://httpbin.org/get"), NULL);
    CFHTTPMessageRef httpMessageRef = CFHTTPMessageCreateRequest(kCFAllocatorDefault, CFSTR("GET"), urlRef, kCFHTTPVersion1_1);
    CFRelease(urlRef);

    CFReadStreamRef readStream = CFReadStreamCreateForHTTPRequest(kCFAllocatorDefault, httpMessageRef);
    CFRelease(httpMessageRef);

    CFReadStreamScheduleWithRunLoop(readStream, CFRunLoopGetCurrent(), kCFRunLoopCommonModes);

    CFOptionFlags eventFlags =(kCFStreamEventHasBytesAvailable | kCFStreamEventErrorOccurred | kCFStreamEventEndEncountered);
    CFStreamClientContext context = {
        0,
        NULL,
        NULL,
        NULL,
       NULL
    } ;
    //Assigns a client to a stream, which receives callbacks when certain events occur.
    CFReadStreamSetClient(readStream, eventFlags, CFNetworkRequestCallback, &context);
    //Opens a stream for reading.
    CFReadStreamOpen(readStream);
}
//callback
void CFNetworkRequestCallback(CFReadStreamRef _Null_unspecified stream, CFStreamEventType type, void * _Null_unspecified clientCallBackInfo) {
    CFMutableDataRef responseBytes = CFDataCreateMutable(kCFAllocatorDefault, 0);
    CFIndex numberOfBytesRead = 0;
    do {
        UInt8 buffer[2014];
        numberOfBytesRead = CFReadStreamRead(stream, buffer, sizeof(buffer));
        if(numberOfBytesRead > 0) {
            CFDataAppendBytes(responseBytes, buffer, numberOfBytesRead);
        }
    } while(numberOfBytesRead > 0);


    CFHTTPMessageRef response =(CFHTTPMessageRef)CFReadStreamCopyProperty(stream, kCFStreamPropertyHTTPResponseHeader);
    if(responseBytes) {
        if(response) {
            CFHTTPMessageSetBody(response, responseBytes);
        }
        CFRelease(responseBytes);
    }

    //close and cleanup
    CFReadStreamClose(stream);
    CFReadStreamUnscheduleFromRunLoop(stream, CFRunLoopGetCurrent(), kCFRunLoopCommonModes);
    CFRelease(stream);

    //print response
    if(response) {
        CFDataRef reponseBodyData = CFHTTPMessageCopyBody(response);
        CFRelease(response);

        printResponseData(reponseBodyData);
        CFRelease(reponseBodyData);
    }
}

void printResponseData(CFDataRef responseData) {
    CFIndex dataLength = CFDataGetLength(responseData);
    UInt8 *bytes =(UInt8 *)malloc(dataLength);
    CFDataGetBytes(responseData, CFRangeMake(0, CFDataGetLength(responseData)), bytes);
    CFStringRef responseString = CFStringCreateWithBytes(kCFAllocatorDefault, bytes, dataLength, kCFStringEncodingUTF8, TRUE);
    CFShow(responseString);
    CFRelease(responseString);
    free(bytes);
}
//console
{
  "args":{}, 
  "headers":{
    "Host":"httpbin.org", 
    "User-Agent":"Test/1 CFNetwork/1125.2 Darwin/19.3.0", 
    "X-Amzn-Trace-Id":"Root=1-5e8980d0-581f3f44724c7140614c2564"
  }, 
  "origin":"183.159.122.102", 
  "url":"https://httpbin.org/get"
}

 NSURLSession NSURLConnection CFNetwork                                                        runtime hook                           hook                       hook                                        NSURLConnection NSURLSession CFNetwork                                                                                           

NSURLSession NSURLConnection hook
NSURLSession Hook
NSURLConnection Hook

APM    CFNetwork           

CFNetwork c c hook Dynamic Loader Hook - fishhook

Dynamic Loader dyld Mach-O Runtime C fishhook __DATA segment __nl_symbol_ptr __la_symbol_ptr section Indirect Symbol Table Symbol Table String Table hook

/* Returns the number of bytes read, or -1 if an error occurs preventing any

bytes from being read, or 0 if the stream's end was encountered.

It is an error to try and read from a stream that hasn't been opened first.

This call will block until at least one byte is available; it will NOT block

until the entire buffer can be filled. To avoid blocking, either poll using

CFReadStreamHasBytesAvailable() or use the run loop and listen for the

kCFStreamEventHasBytesAvailable event for notification of data available. */

CF_EXPORT

CFIndex CFReadStreamRead(CFReadStreamRef _Null_unspecified stream, UInt8 * _Null_unspecified buffer, CFIndex bufferLength);

CFNetwork CFReadStreamRef

           NSURLConnection   
  • Hook method swizzling

    carbon(3).png

  • NSProxy

    carbon(5).png

  • NSURLConnection NSURLSession NSIuputStream

    carbon(6).png

  • NSURLConnection Category hook hook NSURLConnection

    carbon(7).png

                         SDK                  
2.3.2
                          **isa swizzling** 

       NSURLConnection NSURLSession NSInputStream       hook       NSProxy                            **isa swizzling** 
  • Method swizzling

    carbon(12).png
    method swizzling

method swizzling

carbon(13).png

  • isa swizzling

    carbon(14).png
    isa swizzling

         `isa`         
  1. APM

  2. APM NSURLSession NSURLConnection

    KVO

  • getter seeter

  • isa

  • getter setter

  • isa

    NSURLConnection NSURLSession load -(**nullable** **instancetype**)initWithRequest:(NSURLRequest *)request delegate:(**nullable** **id**)delegate startImmediately:(**BOOL**)startImmediately; NSURLSession NSURLConnection isa isa

    isa swizzling method swizzling NSProxy

    isa Demo KVO

carbon(15).png

2.4 App

                        [AFNetworking](https://github.com/AFNetworking/AFNetworking)                     

AFNetworking AFNetworkingTaskDidResumeNotification AFNetworkingTaskDidCompleteNotification

carbon(16).png
networkRecoder

                                                                              **NSURLSessionTask**         runtime                

    Category                                          NSString                               A   NSString             getMaskedIdCardNumber          \[9, 12\] 4               B                      NSString         getMaskedIdCardNumber              \[8, 11\] 4                                                                    NSString            ?    

   SDK            
  • Category SDK +SDK _ SDK JuhuaSuanAPM NSURLSessionTask Category NSURLSessionTask+JuHuaSuanAPM_NetworkMonitor.h
  • Category SDK SDK _ JuhuaSuanAPM_requestId`
  • Category SDK SDK _ -(BOOL)JuhuaSuanAPM__isGzippedData

carbon(17).png

2.5 iOS

2.5.1 HTTP

HTTP

2020-05-16-HTTPResponseStructure.png

  1. HTTP
  2. ASCII 2
  3. HTTP Headers
<method> <request-URI> <version>
<headers>

<entity-body>



<version> <status> <reason-phrase>
<headers>

<entity-body>

  Chrome                                

2020-05-13-HTTPDataStructure.png

     `curl`                 

2020-05-14-HTTPRequestStructure.png

   HTTP             gzip             NSURLProtocol         NSData                            HTTP           gzip                 NSData     
2.5.2
  1. Request Response

       App    Crash      Request   Response               
*                          
*             Cookie        
*                          `HTTPBody.length`       
*                          
*            body               `exceptedContentLength`       
*                  gzip                             `Accept-Encoding`                                                                                    `Content-Encoding`                    
2.5.3
                     NSURLProtocol          Hook                              
2.5.3.1 Request
  1. NSURLProtocol App

  2. NSURLProtocol Socket

    carbon(9).png

    carbon(10).png

  3. Status Line

    NSURLResponse Status Line HTTP Version Status Line CFNetwork API

    ** NSURLResponse _CFURLResponse CFTypeRef CFTypeRef CFHTTPMessageRef CFHTTPMessageCopyResponseStatusLine CFHTTPMessageRef Status Line **

    Status Line         NSURLResponse     

    carbon(11).png

  4. Status Line NSData
    carbon.png

  5. Header

    allHeaderFields NSDictionary key:value NSData

    key:value key curl chrome Network

    carbon.png

  6. Body

    Body excepectedContentLength allHeaderFields Content-Length

    /*!

    @abstract Returns the expected content length of the receiver.

    @discussion Some protocol implementations report a content length

    as part of delivering load metadata, but not all protocols

    guarantee the amount of data that will be delivered in actuality.

    Hence, this method returns an expected amount. Clients should use

    this value as an advisory, and should be prepared to deal with

    either more or less data.

    @result The expected content length of the receiver, or -1 if

    there is no expectation that can be arrived at regarding expected

    content length.

    */

    @property(readonly) long long expectedContentLength;

    • HTTP 1.1 Transfer-Encoding:chunked header Content-Length
    • HTTP 1.0 content-length
    • HTTP 1.1 keep alive Content-Length chunked keep alive HTTP 1.0 Content-Length

    Transfer-Encoding:chunked

            `Content-Length`             .                     ,                  `\r\n` ,        ,      `\r\n`             ,           0.

    NSMutableData stopLoading Body

  • didReceiveData data

    carbon(1).png

  • stopLoading allHeaderFields Content-Encoding key gzip stopLoading NSData gzip gzip

![carbon(2).png](https://i0.wp.com/segmentfault.com/img/bVbINGj "carbon(2).png")
2.5.3.2 Resquest
  1. NSURLProtocol App

  2. NSURLProtocol Socket

    carbon(3).png

    carbon(4).png

  3. Status Line

    NSURLRequest NSURLResponse StatusLine Status Line + + + + +

    NSURLRequest Status Line

    carbon(5).png

  4. Header

    HTTP DNS IP HTTPS TLS IP TCP Cookie

            cookie ?                    
    
        NSURLRequest                                                                      WebView                      
    
      NSURLRequest   `allHeaderFields`      cookie          Header   

    carbon(6).png

  5. Body

    NSURLConnection HTTPBody WebView ajax HTTPBodyStream stream body .

    carbon(7).png

  6. -(NSURLRequest *)connection:(NSURLConnection *)connection willSendRequest:(NSURLRequest *)request redirectResponse:(NSURLResponse *)response

    carbon(8).png


                      App                                   App                  



              `Instrucments`    `Energy Log`                                APM       

1.

iOS IOKit IOKit

  • opensource IOPowerSources.h IOPSKeys.h Xcode Package Contents IOKit.framework /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk/System/Library/Frameworks/IOKit.framework
  • IOPowerSources.h IOPSKeys.h IOKit.framework
  • UIDevice batteryMonitoringEnabled true
  • 1%

2.

   Instrucments    Energy Log          App                   APM                                 

                       dump        

               `thread_basic_info`       CPU           `cpu_usage`                        CPU                      dump                       [3.2](#threadInfo)      

carbon(9).png

3.

CPU CPU CPU GPU CPU GCD dispatch_block_create_with_qos_class(<#dispatch_block_flags_t flags#>, dispatch_qos_class_t qos_class, <#int relative_priority#>, <#^(void)block#>)() qos QOS_CLASS_UTILITY block QOS_CLASS_UTILITY

CPU I/O iOS NSCache

NSCache NSCache -(**void**)cache:(NSCache *)cache willEvictObject:(**id**)obj; I/O I/O I/O

NSCache SDWebImage I/O NSCache

carbon(10).png

                                    NSCache            NSCache       NSCache   `totalCostLimit countLimit`    

-(void)setObject:(ObjectType)obj forKey:(KeyType)key cost:(NSUInteger)g;

Crash

1.

1.1 Mach

Mach Mach

  • Mach

  • Mach -

    Mach - msg_send() msg_recv()

Mach Mach thread_set_exception_ports(<#thread_act_t thread#>, <#exception_mask_t exception_mask#>, <#mach_port_t new_port#>, <#exception_behavior_t behavior#>, <#thread_state_flavor_t new_flavor#>) NULL UDP

                                                                     `KERN_SUCCESS`                Mach                        

1.2 BSD

BSD XUN POSIX UNIX Mach

Mach BSD Mach UNIX Mach

Mach host ux_exception unix threadsignal
Mach           Unix

2. Crash

iOS Apples`s Crash Reporter Crash Crash

Incident Identifier:7FA6736D-09E8-47A1-95EC-76C4522BDE1A
CrashReporter Key:  4e2d36419259f14413c3229e8b7235bcc74847f3
Hardware Model:     iPhone7,1
Process:        CMMonitorExample [3608]
Path:           /var/containers/Bundle/Application/9518A4F4-59B7-44E9-BDDA-9FBEE8CA18E5/CMMonitorExample.app/CMMonitorExample
Identifier:     com.Wacai.CMMonitorExample
Version:        1.0(1)
Code Type:      ARM-64
Parent Process: ? [1]

Date/Time:      2017-01-03 11:43:03.000 +0800
OS Version:     iOS 10.2(14C92)
Report Version: 104

Exception Type: EXC_CRASH(SIGABRT)
Exception Codes:0x00000000 at 0x0000000000000000
Crashed Thread: 0

Application Specific Information:
*** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason:'-[__NSSingleObjectArrayI objectForKey:]:unrecognized selector sent to instance 0x174015060'

Thread 0 Crashed:
0   CoreFoundation                  0x0000000188f291b8 0x188df9000 + 1245624(<redacted> + 124)
1   libobjc.A.dylib                 0x000000018796055c 0x187958000 + 34140(objc_exception_throw + 56)
2   CoreFoundation                  0x0000000188f30268 0x188df9000 + 1274472(<redacted> + 140)
3   CoreFoundation                  0x0000000188f2d270 0x188df9000 + 1262192(<redacted> + 916)
4   CoreFoundation                  0x0000000188e2680c 0x188df9000 + 186380(_CF_forwarding_prep_0 + 92)
5   CMMonitorExample                0x000000010004c618 0x100044000 + 34328(-[MakeCrashHandler throwUncaughtNSException]+ 80)

Crash     `Exception Type`   2     Mach    + Unix    

Exception Type:EXC_CRASH(SIGABRT) Mach EXC_CRASH host SIGABRT

** ** Mach Unix Crash

**** Mach 1.2 Mach Mach Unix

                    KSCrash plcrashreporter          Bugly                                 bug              KSCrash       KSCrash        

KSCrash Crash

  • Mach kernel exceptions

  • Fatal signals

  • C++ exceptions

  • Objective-C exceptions

  • Main thread deadlock(experimental)

  • Custom crashes(e.g. from scripting languages)

    iOS Crash KSCrash Crash

2.1. Mach

                                                                Mach            SDK                                                                 Crash            json    

KSCrash

Mach

     :

Mach

static bool installExceptionHandler()
{
    KSLOG_DEBUG("Installing mach exception handler.");

    bool attributes_created = false;
    pthread_attr_t attr;

    kern_return_t kr;
    int error;
    //     
    const task_t thisTask = mach_task_self();
    exception_mask_t mask = EXC_MASK_BAD_ACCESS |
    EXC_MASK_BAD_INSTRUCTION |
    EXC_MASK_ARITHMETIC |
    EXC_MASK_SOFTWARE |
    EXC_MASK_BREAKPOINT;

    KSLOG_DEBUG("Backing up original exception ports.");
    //   Task           
    kr = task_get_exception_ports(thisTask,
                                  mask,
                                  g_previousExceptionPorts.masks,
                                  &g_previousExceptionPorts.count,
                                  g_previousExceptionPorts.ports,
                                  g_previousExceptionPorts.behaviors,
                                  g_previousExceptionPorts.flavors);
    //     failed   
    if(kr != KERN_SUCCESS)
    {
        KSLOG_ERROR("task_get_exception_ports:%s", mach_error_string(kr));
        goto failed;
    }
    //KSCrash            
    if(g_exceptionPort == MACH_PORT_NULL)
    {
        KSLOG_DEBUG("Allocating new port with receive rights.");
        //       
        kr = mach_port_allocate(thisTask,
                                MACH_PORT_RIGHT_RECEIVE,
                                &g_exceptionPort);
        if(kr != KERN_SUCCESS)
        {
            KSLOG_ERROR("mach_port_allocate:%s", mach_error_string(kr));
            goto failed;
        }

        KSLOG_DEBUG("Adding send rights to port.");
        //           MACH_MSG_TYPE_MAKE_SEND
        kr = mach_port_insert_right(thisTask,
                                    g_exceptionPort,
                                    g_exceptionPort,
                                    MACH_MSG_TYPE_MAKE_SEND);
        if(kr != KERN_SUCCESS)
        {
            KSLOG_ERROR("mach_port_insert_right:%s", mach_error_string(kr));
            goto failed;
        }
    }

    KSLOG_DEBUG("Installing port as exception handler.");
    //  Task         
    kr = task_set_exception_ports(thisTask,
                                  mask,
                                  g_exceptionPort,
                                  EXCEPTION_DEFAULT,
                                  THREAD_STATE_NONE);
    if(kr != KERN_SUCCESS)
    {
        KSLOG_ERROR("task_set_exception_ports:%s", mach_error_string(kr));
        goto failed;
    }

    KSLOG_DEBUG("Creating secondary exception thread(suspended).");
    pthread_attr_init(&attr);
    attributes_created = true;
    pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_DETACHED);
    //     
    error = pthread_create(&g_secondaryPThread,
                           &attr,
                           &handleExceptions,
                           kThreadSecondary);
    if(error != 0)
    {
        KSLOG_ERROR("pthread_create_suspended_np:%s", strerror(error));
        goto failed;
    }
    //   Mach     
    g_secondaryMachThread = pthread_mach_thread_np(g_secondaryPThread);
    ksmc_addReservedThread(g_secondaryMachThread);

    KSLOG_DEBUG("Creating primary exception thread.");
    error = pthread_create(&g_primaryPThread,
                           &attr,
                           &handleExceptions,
                           kThreadPrimary);
    if(error != 0)
    {
        KSLOG_ERROR("pthread_create:%s", strerror(error));
        goto failed;
    }
    pthread_attr_destroy(&attr);
    g_primaryMachThread = pthread_mach_thread_np(g_primaryPThread);
    ksmc_addReservedThread(g_primaryMachThread);

    KSLOG_DEBUG("Mach exception handler installed.");
    return true;


failed:
    KSLOG_DEBUG("Failed to install mach exception handler.");
    if(attributes_created)
    {
        pthread_attr_destroy(&attr);
    }
    //                 
    uninstallExceptionHandler();
    return false;
}



/** Our exception handler thread routine.
 * Wait for an exception message, uninstall our exception port, record the
 * exception information, and write a report.
 */
static void* handleExceptions(void* const userData)
{
    MachExceptionMessage exceptionMessage = {{0}};
    MachReplyMessage replyMessage = {{0}};
    char* eventID = g_primaryEventID;

    const char* threadName =(const char*) userData;
    pthread_setname_np(threadName);
    if(threadName == kThreadSecondary)
    {
        KSLOG_DEBUG("This is the secondary thread. Suspending.");
        thread_suspend((thread_t)ksthread_self());
        eventID = g_secondaryEventID;
    }
    //             
    for(;;)
    {
        KSLOG_DEBUG("Waiting for mach exception");

        //Wait for a message.
        kern_return_t kr = mach_msg(&exceptionMessage.header,
                                    MACH_RCV_MSG,
                                    0,
                                    sizeof(exceptionMessage),
                                    g_exceptionPort,
                                    MACH_MSG_TIMEOUT_NONE,
                                    MACH_PORT_NULL);
        //            Mach        for        
        if(kr == KERN_SUCCESS)
        {
            break;
        }

        //Loop and try again on failure.
        KSLOG_ERROR("mach_msg:%s", mach_error_string(kr));
    }

    KSLOG_DEBUG("Trapped mach exception code 0x%x, subcode 0x%x",
                exceptionMessage.code[0], exceptionMessage.code[1]);
    if(g_isEnabled)
    {
        //     
        ksmc_suspendEnvironment();
        g_isHandlingCrash = true;
        //      
        kscm_notifyFatalExceptionCaptured(true);

        KSLOG_DEBUG("Exception handler is installed. Continuing exception handling.");


        //Switch to the secondary thread if necessary, or uninstall the handler
        //to avoid a death loop.
        if(ksthread_self() == g_primaryMachThread)
        {
            KSLOG_DEBUG("This is the primary exception thread. Activating secondary thread.");
//TODO:This was put here to avoid a freeze. Does secondary thread ever fire?
            restoreExceptionPorts();
            if(thread_resume(g_secondaryMachThread) != KERN_SUCCESS)
            {
                KSLOG_DEBUG("Could not activate secondary thread. Restoring original exception ports.");
            }
        }
        else
        {
            KSLOG_DEBUG("This is the secondary exception thread. Restoring original exception ports.");
//          restoreExceptionPorts();
        }

        //Fill out crash information
        //             
        KSLOG_DEBUG("Fetching machine state.");
        KSMC_NEW_CONTEXT(machineContext);
        KSCrash_MonitorContext* crashContext = &g_monitorContext;
        crashContext->offendingMachineContext = machineContext;
        kssc_initCursor(&g_stackCursor, NULL, NULL);
        if(ksmc_getContextForThread(exceptionMessage.thread.name, machineContext, true))
        {
            kssc_initWithMachineContext(&g_stackCursor, 100, machineContext);
            KSLOG_TRACE("Fault address 0x%x, instruction address 0x%x", kscpu_faultAddress(machineContext), kscpu_instructionAddress(machineContext));
            if(exceptionMessage.exception == EXC_BAD_ACCESS)
            {
                crashContext->faultAddress = kscpu_faultAddress(machineContext);
            }
            else
            {
                crashContext->faultAddress = kscpu_instructionAddress(machineContext);
            }
        }

        KSLOG_DEBUG("Filling out context.");
        crashContext->crashType = KSCrashMonitorTypeMachException;
        crashContext->eventID = eventID;
        crashContext->registersAreValid = true;
        crashContext->mach.type = exceptionMessage.exception;
        crashContext->mach.code = exceptionMessage.code[0];
        crashContext->mach.subcode = exceptionMessage.code[1];
        if(crashContext->mach.code == KERN_PROTECTION_FAILURE && crashContext->isStackOverflow)
        {
            //A stack overflow should return KERN_INVALID_ADDRESS, but
            //when a stack blasts through the guard pages at the top of the stack,
            //it generates KERN_PROTECTION_FAILURE. Correct for this.
            crashContext->mach.code = KERN_INVALID_ADDRESS;
        }
        crashContext->signal.signum = signalForMachException(crashContext->mach.type, crashContext->mach.code);
        crashContext->stackCursor = &g_stackCursor;

        kscm_handleException(crashContext);

        KSLOG_DEBUG("Crash handling complete. Restoring original handlers.");
        g_isHandlingCrash = false;
        ksmc_resumeEnvironment();
    }

    KSLOG_DEBUG("Replying to mach exception message.");
    //Send a reply saying "I didn't handle this exception".
    replyMessage.header = exceptionMessage.header;
    replyMessage.NDR = exceptionMessage.NDR;
    replyMessage.returnCode = KERN_FAILURE;

    mach_msg(&replyMessage.header,
             MACH_SEND_MSG,
             sizeof(replyMessage),
             0,
             MACH_PORT_NULL,
             MACH_MSG_TIMEOUT_NONE,
             MACH_PORT_NULL);

    return NULL;
}



/** Restore the original mach exception ports.
 */
static void restoreExceptionPorts(void)
{
    KSLOG_DEBUG("Restoring original exception ports.");
    if(g_previousExceptionPorts.count == 0)
    {
        KSLOG_DEBUG("Original exception ports were already restored.");
        return;
    }

    const task_t thisTask = mach_task_self();
    kern_return_t kr;

    //Reinstall old exception ports.
    //for           KSCrash                     
    for(mach_msg_type_number_t i = 0; i < g_previousExceptionPorts.count; i++)
    {
        KSLOG_TRACE("Restoring port index%d", i);
        kr = task_set_exception_ports(thisTask,
                                      g_previousExceptionPorts.masks[i],
                                      g_previousExceptionPorts.ports[i],
                                      g_previousExceptionPorts.behaviors[i],
                                      g_previousExceptionPorts.flavors[i]);
        if(kr != KERN_SUCCESS)
        {
            KSLOG_ERROR("task_set_exception_ports:%s",
                        mach_error_string(kr));
        }
    }
    KSLOG_DEBUG("Exception ports restored.");
    g_previousExceptionPorts.count = 0;
}

2.2. Signal

Mach Unix signanHandler

KSCrash
2020-05-20-signalCrash.png

   :



static bool installSignalHandler()
{
    KSLOG_DEBUG("Installing signal handler.");

#if KSCRASH_HAS_SIGNAL_STACK
    //         
    if(g_signalStack.ss_size == 0)
    {
        KSLOG_DEBUG("Allocating signal stack area.");
        g_signalStack.ss_size = SIGSTKSZ;
        g_signalStack.ss_sp = malloc(g_signalStack.ss_size);
    }
    //                       
    //sigaltstack()          1     sigstack     stack_t                                   2     old_sigstack      stack_t                          (  )
    KSLOG_DEBUG("Setting signal stack area.");
    //sigaltstack                             NULL     NULL                             0     -1.
    if(sigaltstack(&g_signalStack, NULL) != 0)
    {
        KSLOG_ERROR("signalstack:%s", strerror(errno));
        goto failed;
    }
#endif

    const int* fatalSignals = kssignal_fatalSignals();
    int fatalSignalsCount = kssignal_numFatalSignals();

    if(g_previousSignalHandlers == NULL)
    {
        KSLOG_DEBUG("Allocating memory to store previous signal handlers.");
        g_previousSignalHandlers = malloc(sizeof(*g_previousSignalHandlers)
                                          *(unsigned)fatalSignalsCount);
    }

    //        sigaction            sigaction    
    struct sigaction action = {{0}};
    //sa_flags      SA_ONSTACK                                  
    action.sa_flags = SA_SIGINFO | SA_ONSTACK;
#if KSCRASH_HOST_APPLE && defined(__LP64__)
    action.sa_flags |= SA_64REGSET;
#endif
    sigemptyset(&action.sa_mask);
    action.sa_sigaction = &handleSignal;

    //          
    for(int i = 0; i < fatalSignalsCount; i++)
    {
        //                  action       g_previousSignalHandlers            
        KSLOG_DEBUG("Assigning handler for signal%d", fatalSignals[i]);
        if(sigaction(fatalSignals[i], &action, &g_previousSignalHandlers[i]) != 0)
        {
            char sigNameBuff[30];
            const char* sigName = kssignal_signalName(fatalSignals[i]);
            if(sigName == NULL)
            {
                snprintf(sigNameBuff, sizeof(sigNameBuff), "%d", fatalSignals[i]);
                sigName = sigNameBuff;
            }
            KSLOG_ERROR("sigaction(%s):%s", sigName, strerror(errno));
            //Try to reverse the damage
            for(i--;i >= 0; i--)
            {
                sigaction(fatalSignals[i], &g_previousSignalHandlers[i], NULL);
            }
            goto failed;
        }
    }
    KSLOG_DEBUG("Signal handlers installed.");
    return true;

failed:
    KSLOG_DEBUG("Failed to install signal handlers.");
    return false;
}



static void handleSignal(int sigNum, siginfo_t* signalInfo, void* userContext)
{
    KSLOG_DEBUG("Trapped signal%d", sigNum);
    if(g_isEnabled)
    {
        ksmc_suspendEnvironment();
        kscm_notifyFatalExceptionCaptured(false);

        KSLOG_DEBUG("Filling out context.");
        KSMC_NEW_CONTEXT(machineContext);
        ksmc_getContextForSignal(userContext, machineContext);
        kssc_initWithMachineContext(&g_stackCursor, 100, machineContext);
        //            
        KSCrash_MonitorContext* crashContext = &g_monitorContext;
        memset(crashContext, 0, sizeof(*crashContext));
        crashContext->crashType = KSCrashMonitorTypeSignal;
        crashContext->eventID = g_eventID;
        crashContext->offendingMachineContext = machineContext;
        crashContext->registersAreValid = true;
        crashContext->faultAddress =(uintptr_t)signalInfo->si_addr;
        crashContext->signal.userContext = userContext;
        crashContext->signal.signum = signalInfo->si_signo;
        crashContext->signal.sigcode = signalInfo->si_code;
        crashContext->stackCursor = &g_stackCursor;

        kscm_handleException(crashContext);
        ksmc_resumeEnvironment();
    }

    KSLOG_DEBUG("Re-raising signal for regular handlers to catch.");
    //This is technically not allowed, but it works in OSX and iOS.
    raise(sigNum);
}

KSCrash

static void uninstallSignalHandler(void)
{
    KSLOG_DEBUG("Uninstalling signal handlers.");

    const int* fatalSignals = kssignal_fatalSignals();
    int fatalSignalsCount = kssignal_numFatalSignals();
    //                      
    for(int i = 0; i < fatalSignalsCount; i++)
    {
        KSLOG_DEBUG("Restoring original handler for signal%d", fatalSignals[i]);
        sigaction(fatalSignals[i], &g_previousSignalHandlers[i], NULL);
    }

    KSLOG_DEBUG("Signal handlers uninstalled.");
}
  1. n

  2. int sigaltstack(const stack_t * __restrict, stack_t * __restrict) stack_t 1 2 ( )

    _STRUCT_SIGALTSTACK
    {
        void            *ss_sp;         /* signal stack base */
        __darwin_size_t ss_size;        /* signal stack length */
        int             ss_flags;       /* SA_DISABLE and/or SA_ONSTACK */
    };
    typedef _STRUCT_SIGALTSTACK     stack_t; /* [???]signal stack */
    
           `ss_flags`       0       `SIGSTKSZ`                    

    carbon(12).png

    sigaltstack

    ss_flags SS_ONSTACK EPERM( ) SS_DISABLE

  3. int sigaction(int, const struct sigaction * __restrict, struct sigaction * __restrict);

                     `SIGKILL`   `SIGSTOP`                                         `SIGKILL` and `SIGSTOP` cannot be caught, blocked, or ignored  
    
             `sigaction`                                                                                           

    carbon(13).png

    sigaction sa_flags SA_ONSTACK

2.3. C++

c++ std::set_terminate(CPPExceptionTerminate)

iOS C C++ C++ NSException OC C++ default_terminate_handler C++ terminate abort_message abort SIGABRT

  C++         `try...catch...`               `NSException`       C++                          `SIGABRT`                         

why? try...catch... __cxa_rethrow() __cxa_rethrow() unwind unwind catch catch C++

static void setEnabled(bool isEnabled)
{
    if(isEnabled != g_isEnabled)
    {
        g_isEnabled = isEnabled;
        if(isEnabled)
        {
            initialize();

            ksid_generate(g_eventID);
            g_originalTerminateHandler = std::set_terminate(CPPExceptionTerminate);
        }
        else
        {
            std::set_terminate(g_originalTerminateHandler);
        }
        g_captureNextStackTrace = isEnabled;
    }
}

static void initialize()
{
    static bool isInitialized = false;
    if(!isInitialized)
    {
        isInitialized = true;
        kssc_initCursor(&g_stackCursor, NULL, NULL);
    }
}

void kssc_initCursor(KSStackCursor *cursor,
                     void(*resetCursor)(KSStackCursor*),
                     bool(*advanceCursor)(KSStackCursor*))
{
    cursor->symbolicate = kssymbolicator_symbolicate;
    cursor->advanceCursor = advanceCursor != NULL ? advanceCursor :g_advanceCursor;
    cursor->resetCursor = resetCursor != NULL ? resetCursor :kssc_resetCursor;
    cursor->resetCursor(cursor);
}

carbon(11).png

2.4. Objective-C

OC NSException NSUncaughtExceptionHandler NSException Crash

static void setEnabled(bool isEnabled)
{
    if(isEnabled != g_isEnabled)
    {
        g_isEnabled = isEnabled;
        if(isEnabled)
        {
            KSLOG_DEBUG(@"Backing up original handler.");
            //     OC       
            g_previousUncaughtExceptionHandler = NSGetUncaughtExceptionHandler();

            KSLOG_DEBUG(@"Setting new handler.");
            //    OC       
            NSSetUncaughtExceptionHandler(&handleException);
            KSCrash.sharedInstance.uncaughtExceptionHandler = &handleException;
        }
        else
        {
            KSLOG_DEBUG(@"Restoring original handler.");
            NSSetUncaughtExceptionHandler(g_previousUncaughtExceptionHandler);
        }
    }
}

2.5.

      ANR        
  • do...while... autorelease
  • awaitingResponse watchdogPulse watchdogPulse awaitingResponse YES awaitingResponse NO

carbon(15).png

  • g_watchdogInterval awaitingResponse

    carbon(16).png

2.6 Crash

2.6.1 Crash
     iOS          crash                crash        crash                    

     crash        KSCrash       crash     

//KSCrashMonitor_Deadlock.m
-(void) handleDeadlock
{
    ksmc_suspendEnvironment();
    kscm_notifyFatalExceptionCaptured(false);

    KSMC_NEW_CONTEXT(machineContext);
    ksmc_getContextForThread(g_mainQueueThread, machineContext, false);
    KSStackCursor stackCursor;
    kssc_initWithMachineContext(&stackCursor, 100, machineContext);
    char eventID[37];
    ksid_generate(eventID);

    KSLOG_DEBUG(@"Filling out context.");
    KSCrash_MonitorContext* crashContext = &g_monitorContext;
    memset(crashContext, 0, sizeof(*crashContext));
    crashContext->crashType = KSCrashMonitorTypeMainThreadDeadlock;
    crashContext->eventID = eventID;
    crashContext->registersAreValid = false;
    crashContext->offendingMachineContext = machineContext;
    crashContext->stackCursor = &stackCursor;

    kscm_handleException(crashContext);
    ksmc_resumeEnvironment();

    KSLOG_DEBUG(@"Calling abort()");
    abort();
}

 crash                 `kscm_handleException()`                    crash          

2020-06-03-KSCrashCaller.png

/** Start general exception processing.
 *
 * @oaram context Contextual information about the exception.
 */
void kscm_handleException(struct KSCrash_MonitorContext* context)
{
    context->requiresAsyncSafety = g_requiresAsyncSafety;
    if(g_crashedDuringExceptionHandling)
    {
        context->crashedDuringCrashHandling = true;
    }
    for(int i = 0; i < g_monitorsCount; i++)
    {
        Monitor* monitor = &g_monitors[i];
        //     crash        
        if(isMonitorEnabled(monitor))
        {
            //    crash             
            addContextualInfoToEvent(monitor, context);
        }
    }
    //    crash       json     crash   
    g_onExceptionEvent(context);


    if(g_handlingFatalException && !g_crashedDuringExceptionHandling)
    {
        KSLOG_DEBUG("Exception is fatal. Restoring original handlers.");
        kscm_setActiveMonitors(KSCrashMonitorTypeNone);
    }
}

g_onExceptionEvent block static void(*g_onExceptionEvent)(struct KSCrash_MonitorContext* monitorContext); KSCrashMonitor.c

void kscm_setEventCallback(void(*onEvent)(struct KSCrash_MonitorContext* monitorContext))
{
    g_onExceptionEvent = onEvent;
}

kscm_setEventCallback() KSCrashC.c

KSCrashMonitorType kscrash_install(const char* appName, const char* const installPath)
{
    KSLOG_DEBUG("Installing crash reporter.");

    if(g_installed)
    {
        KSLOG_DEBUG("Crash reporter already installed.");
        return g_monitoring;
    }
    g_installed = 1;

    char path[KSFU_MAX_PATH_LENGTH];
    snprintf(path, sizeof(path), "%s/Reports", installPath);
    ksfu_makePath(path);
    kscrs_initialize(appName, path);

    snprintf(path, sizeof(path), "%s/Data", installPath);
    ksfu_makePath(path);
    snprintf(path, sizeof(path), "%s/Data/CrashState.json", installPath);
    kscrashstate_initialize(path);

    snprintf(g_consoleLogPath, sizeof(g_consoleLogPath), "%s/Data/ConsoleLog.txt", installPath);
    if(g_shouldPrintPreviousLog)
    {
        printPreviousLog(g_consoleLogPath);
    }
    kslog_setLogFilename(g_consoleLogPath, true);

    ksccd_init(60);
    //  crash      callback   
    kscm_setEventCallback(onCrash);
    KSCrashMonitorType monitors = kscrash_setMonitoring(g_monitoring);

    KSLOG_DEBUG("Installation complete.");
    return monitors;
}

/** Called when a crash occurs.
 *
 * This function gets passed as a callback to a crash handler.
 */
static void onCrash(struct KSCrash_MonitorContext* monitorContext)
{
    KSLOG_DEBUG("Updating application state to note crash.");
    kscrashstate_notifyAppCrash();
    monitorContext->consoleLogPath = g_shouldAddConsoleLogToReport ? g_consoleLogPath :NULL;

    //    crash           crash
    if(monitorContext->crashedDuringCrashHandling)
    {
        kscrashreport_writeRecrashReport(monitorContext, g_lastCrashReportFilePath);
    }
    else
    {
        //1.             crash      
        char crashReportFilePath[KSFU_MAX_PATH_LENGTH];
        kscrs_getNextCrashReportPath(crashReportFilePath);
        //2.              g_lastCrashReportFilePath
        strncpy(g_lastCrashReportFilePath, crashReportFilePath, sizeof(g_lastCrashReportFilePath));
        //3.                 crash   
        kscrashreport_writeStandardReport(monitorContext, crashReportFilePath);
    }
}

                 2                 json              crash           crash             `kscrashreport_writeRecrashReport()`            `kscrashreport_writeStandardReport()` 

bool ksfu_openBufferedWriter(KSBufferedWriter* writer, const char* const path, char* writeBuffer, int writeBufferLength)
{
    writer->buffer = writeBuffer;
    writer->bufferLength = writeBufferLength;
    writer->position = 0;
    /*
     open()                  
     #define O_RDONLY        0x0000         open for reading only
     #define O_WRONLY        0x0001         open for writing only
     #define O_RDWR          0x0002         open for reading and writing
     #define O_ACCMODE       0x0003         mask for above mode

     #define O_CREAT         0x0200         create if nonexistant
     #define O_TRUNC         0x0400         truncate to zero length
     #define O_EXCL          0x0800         error if already exists

     0755      //                  
     0644                          
                       -1
     */
    writer->fd = open(path, O_RDWR | O_CREAT | O_EXCL, 0644);
    if(writer->fd < 0)
    {
        KSLOG_ERROR("Could not open crash report file%s:%s", path, strerror(errno));
        return false;
    }
    return true;
}

/**
 * Write a standard crash report to a file.
 *
 *  @param monitorContext Contextual information about the crash and environment.
 *                      The caller must fill this out before passing it in.
 *
 *  @param path The file to write to.
 */
void kscrashreport_writeStandardReport(const struct KSCrash_MonitorContext* const monitorContext,
                                       const char* path)
{
        KSLOG_INFO("Writing crash report to%s", path);
    char writeBuffer[1024];
    KSBufferedWriter bufferedWriter;

    if(!ksfu_openBufferedWriter(&bufferedWriter, path, writeBuffer, sizeof(writeBuffer)))
    {
        return;
    }

    ksccd_freeze();

    KSJSONEncodeContext jsonContext;
    jsonContext.userData = &bufferedWriter;
    KSCrashReportWriter concreteWriter;
    KSCrashReportWriter* writer = &concreteWriter;
    prepareReportWriter(writer, &jsonContext);

    ksjson_beginEncode(getJsonContext(writer), true, addJSONData, &bufferedWriter);

    writer->beginObject(writer, KSCrashField_Report);
    {
        writeReportInfo(writer,
                        KSCrashField_Report,
                        KSCrashReportType_Standard,
                        monitorContext->eventID,
                        monitorContext->System.processName);
        ksfu_flushBufferedWriter(&bufferedWriter);

        writeBinaryImages(writer, KSCrashField_BinaryImages);
        ksfu_flushBufferedWriter(&bufferedWriter);

        writeProcessState(writer, KSCrashField_ProcessState, monitorContext);
        ksfu_flushBufferedWriter(&bufferedWriter);

        writeSystemInfo(writer, KSCrashField_System, monitorContext);
        ksfu_flushBufferedWriter(&bufferedWriter);

        writer->beginObject(writer, KSCrashField_Crash);
        {
            writeError(writer, KSCrashField_Error, monitorContext);
            ksfu_flushBufferedWriter(&bufferedWriter);
            writeAllThreads(writer,
                            KSCrashField_Threads,
                            monitorContext,
                            g_introspectionRules.enabled);
            ksfu_flushBufferedWriter(&bufferedWriter);
        }
        writer->endContainer(writer);

        if(g_userInfoJSON != NULL)
        {
            addJSONElement(writer, KSCrashField_User, g_userInfoJSON, false);
            ksfu_flushBufferedWriter(&bufferedWriter);
        }
        else
        {
            writer->beginObject(writer, KSCrashField_User);
        }
        if(g_userSectionWriteCallback != NULL)
        {
            ksfu_flushBufferedWriter(&bufferedWriter);
            g_userSectionWriteCallback(writer);
        }
        writer->endContainer(writer);
        ksfu_flushBufferedWriter(&bufferedWriter);

        writeDebugInfo(writer, KSCrashField_Debug, monitorContext);
    }
    writer->endContainer(writer);

    ksjson_endEncode(getJsonContext(writer));
    ksfu_closeBufferedWriter(&bufferedWriter);
    ksccd_unfreeze();
}

/** Write a minimal crash report to a file.
 *
 * @param monitorContext Contextual information about the crash and environment.
 *                       The caller must fill this out before passing it in.
 *
 * @param path The file to write to.
 */
void kscrashreport_writeRecrashReport(const struct KSCrash_MonitorContext* const monitorContext,
                                      const char* path)
{
  char writeBuffer[1024];
    KSBufferedWriter bufferedWriter;
    static char tempPath[KSFU_MAX_PATH_LENGTH];
    //        crash report       /var/mobile/Containers/Data/Application/******/Library/Caches/KSCrash/Test/Reports/Test-report-******.json       .json     .old          /var/mobile/Containers/Data/Application/******/Library/Caches/KSCrash/Test/Reports/Test-report-******.old

    strncpy(tempPath, path, sizeof(tempPath) - 10);
    strncpy(tempPath + strlen(tempPath) - 5, ".old", 5);
    KSLOG_INFO("Writing recrash report to%s", path);

    if(rename(path, tempPath) < 0)
    {
        KSLOG_ERROR("Could not rename%s to%s:%s", path, tempPath, strerror(errno));
    }
    //                 
    if(!ksfu_openBufferedWriter(&bufferedWriter, path, writeBuffer, sizeof(writeBuffer)))
    {
        return;
    }

    ksccd_freeze();
    //json     c   
    KSJSONEncodeContext jsonContext;
    jsonContext.userData = &bufferedWriter;
    KSCrashReportWriter concreteWriter;
    KSCrashReportWriter* writer = &concreteWriter;
    prepareReportWriter(writer, &jsonContext);

    ksjson_beginEncode(getJsonContext(writer), true, addJSONData, &bufferedWriter);

    writer->beginObject(writer, KSCrashField_Report);
    {
        writeRecrash(writer, KSCrashField_RecrashReport, tempPath);
        ksfu_flushBufferedWriter(&bufferedWriter);
        if(remove(tempPath) < 0)
        {
            KSLOG_ERROR("Could not remove%s:%s", tempPath, strerror(errno));
        }
        writeReportInfo(writer,
                        KSCrashField_Report,
                        KSCrashReportType_Minimal,
                        monitorContext->eventID,
                        monitorContext->System.processName);
        ksfu_flushBufferedWriter(&bufferedWriter);

        writer->beginObject(writer, KSCrashField_Crash);
        {
            writeError(writer, KSCrashField_Error, monitorContext);
            ksfu_flushBufferedWriter(&bufferedWriter);
            int threadIndex = ksmc_indexOfThread(monitorContext->offendingMachineContext,
                                                 ksmc_getThreadFromContext(monitorContext->offendingMachineContext));
            writeThread(writer,
                        KSCrashField_CrashedThread,
                        monitorContext,
                        monitorContext->offendingMachineContext,
                        threadIndex,
                        false);
            ksfu_flushBufferedWriter(&bufferedWriter);
        }
        writer->endContainer(writer);
    }
    writer->endContainer(writer);

    ksjson_endEncode(getJsonContext(writer));
    ksfu_closeBufferedWriter(&bufferedWriter);
    ksccd_unfreeze();
}
2.6.2 Crash

App Crash KSCrash App App crash

App

[KSCrashInstallation sendAllReportsWithCompletion:] -> [KSCrash sendAllReportsWithCompletion:] -> [KSCrash allReports] -> [KSCrash reportWithIntID:] ->[KSCrash loadCrashReportJSONWithID:] -> kscrs_readReport

sendAllReportsWithCompletion Crash

//                       crash      
static int getReportCount()
{
    int count = 0;
    DIR* dir = opendir(g_reportsPath);
    if(dir == NULL)
    {
        KSLOG_ERROR("Could not open directory%s", g_reportsPath);
        goto done;
    }
    struct dirent* ent;
    while((ent = readdir(dir)) != NULL)
    {
        if(getReportIDFromFilename(ent->d_name) > 0)
        {
            count++;
        }
    }

done:
    if(dir != NULL)
    {
        closedir(dir);
    }
    return count;
}

//  crash                                    reportID     reportID      crash              
-(NSArray*) allReports
{
    int reportCount = kscrash_getReportCount();
    int64_t reportIDs[reportCount];
    reportCount = kscrash_getReportIDs(reportIDs, reportCount);
    NSMutableArray* reports = [NSMutableArray arrayWithCapacity:(NSUInteger)reportCount];
    for(int i = 0; i < reportCount; i++)
    {
        NSDictionary* report = [self reportWithIntID:reportIDs[i]];
        if(report != nil)
        {
            [reports addObject:report];
        }
    }

    return reports;
}

//   reportID    crash   
-(NSDictionary*) reportWithIntID:(int64_t) reportID
{
    NSData* jsonData = [self loadCrashReportJSONWithID:reportID];
    if(jsonData == nil)
    {
        return nil;
    }

    NSError* error = nil;
    NSMutableDictionary* crashReport = [KSJSONCodec decode:jsonData
                                                   options:KSJSONDecodeOptionIgnoreNullInArray |
                                                           KSJSONDecodeOptionIgnoreNullInObject |
                                                           KSJSONDecodeOptionKeepPartialObject
                                                     error:&error];
    if(error != nil)
    {
        KSLOG_ERROR(@"Encountered error loading crash report%" PRIx64 ":%@", reportID, error);
    }
    if(crashReport == nil)
    {
        KSLOG_ERROR(@"Could not load crash report");
        return nil;
    }
    [self doctorReport:crashReport];

    return crashReport;
}

//reportID    crash        NSData   
-(NSData*) loadCrashReportJSONWithID:(int64_t) reportID
{
    char* report = kscrash_readReport(reportID);
    if(report != NULL)
    {
        return [NSData dataWithBytesNoCopy:report length:strlen(report) freeWhenDone:YES];
    }
    return nil;
}

//reportID    crash     char   
char* kscrash_readReport(int64_t reportID)
{
    if(reportID <= 0)
    {
        KSLOG_ERROR("Report ID was%" PRIx64, reportID);
        return NULL;
    }

    char* rawReport = kscrs_readReport(reportID);
    if(rawReport == NULL)
    {
        KSLOG_ERROR("Failed to load report ID%" PRIx64, reportID);
        return NULL;
    }

    char* fixedReport = kscrf_fixupCrashReport(rawReport);
    if(fixedReport == NULL)
    {
        KSLOG_ERROR("Failed to fixup report ID%" PRIx64, reportID);
    }

    free(rawReport);
    return fixedReport;
}

//        reportID    c    getCrashReportPathByID        path        ksfu_readEntireFile    crash     result
char* kscrs_readReport(int64_t reportID)
{
    pthread_mutex_lock(&g_mutex);
    char path[KSCRS_MAX_PATH_LENGTH];
    getCrashReportPathByID(reportID, path);
    char* result;
    ksfu_readEntireFile(path, &result, NULL, 2000000);
    pthread_mutex_unlock(&g_mutex);
    return result;
}

int kscrash_getReportIDs(int64_t* reportIDs, int count)
{
    return kscrs_getReportIDs(reportIDs, count);
}

int kscrs_getReportIDs(int64_t* reportIDs, int count)
{
    pthread_mutex_lock(&g_mutex);
    count = getReportIDs(reportIDs, count);
    pthread_mutex_unlock(&g_mutex);
    return count;
}
//            ent->d_name    getReportIDFromFilename        reportID         
static int getReportIDs(int64_t* reportIDs, int count)
{
    int index = 0;
    DIR* dir = opendir(g_reportsPath);
    if(dir == NULL)
    {
        KSLOG_ERROR("Could not open directory%s", g_reportsPath);
        goto done;
    }

    struct dirent* ent;
    while((ent = readdir(dir)) != NULL && index < count)
    {
        int64_t reportID = getReportIDFromFilename(ent->d_name);
        if(reportID > 0)
        {
            reportIDs[index++]= reportID;
        }
    }

    qsort(reportIDs,(unsigned)count, sizeof(reportIDs[0]), compareInt64);

done:
    if(dir != NULL)
    {
        closedir(dir);
    }
    return index;
}

//sprintf( 1    2)      2       1       sscanf( 1    2    3)         1        2         3  crash       "App  -report-reportID.json"
static int64_t getReportIDFromFilename(const char* filename)
{
    char scanFormat[100];
    sprintf(scanFormat, "%s-report-%%" PRIx64 ".json", g_appName);

    int64_t reportID = 0;
    sscanf(filename, scanFormat, &reportID);
    return reportID;
}

KSCrash    Crash

2.7 js Crash

2.7.1 JavascriptCore
         JSContext     exceptionHandler              

jsContext.exceptionHandler = ^(JSContext *context, JSValue *exception) {
    //  jscore            
};
2.7.2 h5

h5 Javascript window ErrorEvent error window.onerror()

window.onerror = function(msg, url, lineNumber, columnNumber, error) {
   //     
};

h5

2.7.3 React Native
        RN Demo      Debug Text                    crash

<Text style={styles.sectionTitle} onPress={()=>{1+qw;}}>Debug</Text>

1

iOS    debug      RN             

  `command + d`         Debug    Chrome      Mac      `Command + Option + J`               React      RN       

2020-06-19-ReactNativeCrashMonitor.png

crash stack          sourceMap     

Tips RN Release

  • release_iOS

  • react-native bundle --entry-file index.js --platform ios --dev false --bundle-output release_ios/main.jsbundle --assets-dest release_iOS --sourcemap-output release_ios/index.ios.map;

  • release_iOS .jsbundle assets iOS

    2

    iOS release RN

    iOS crash

    iOS !

2020-06-22-RNUncaughtCrash.png

2020-06-22 22:26:03.318 [info][tid:main][RCTRootView.m:294]Running application todos({
    initialProps =     {
    };
    rootTag = 1;
})
2020-06-22 22:26:03.490 [info][tid:com.facebook.react.JavaScript]Running "todos" with {"rootTag":1,"initialProps":{}}
2020-06-22 22:27:38.673 [error][tid:com.facebook.react.JavaScript]ReferenceError:Can't find variable:qw
2020-06-22 22:27:38.675 [fatal][tid:com.facebook.react.ExceptionsManagerQueue]Unhandled JS Exception:ReferenceError:Can't find variable:qw
2020-06-22 22:27:38.691300+0800 todos[16790:314161]*** Terminating app due to uncaught exception 'RCTFatalException:Unhandled JS Exception:ReferenceError:Can't find variable:qw', reason:'Unhandled JS Exception:ReferenceError:Can't find variable:qw, stack:
onPress@397:1821
<unknown>@203:3896
_performSideEffectsForTransition@210:9689
_performSideEffectsForTransition@(null):(null)
_receiveSignal@210:8425
_receiveSignal@(null):(null)
touchableHandleResponderRelease@210:5671
touchableHandleResponderRelease@(null):(null)
onResponderRelease@203:3006
b@97:1125
S@97:1268
w@97:1322
R@97:1617
M@97:2401
forEach@(null):(null)
U@97:2201
<unknown>@97:13818
Pe@97:90199
Re@97:13478
Ie@97:13664
receiveTouches@97:14448
value@27:3544
<unknown>@27:840
value@27:2798
value@27:812
value@(null):(null)
'
*** First throw call stack:

(
0 CoreFoundation 0x00007fff23e3cf0e exceptionPreprocess + 350
1 libobjc.A.dylib 0x00007fff50ba89b2 objc_exception_throw + 48
2 todos 0x00000001017b0510 RCTFormatError + 0
3 todos 0x000000010182d8ca -[RCTExceptionsManager reportFatal:stack:exceptionId:suppressRedBox:]+ 503
4 todos 0x000000010182e34e -[RCTExceptionsManager reportException:]+ 1658
5 CoreFoundation 0x00007fff23e43e8c __invoking_
+ 140
6 CoreFoundation 0x00007fff23e41071 -[NSInvocation invoke]+ 321
7 CoreFoundation 0x00007fff23e41344 -[NSInvocation invokeWithTarget:]+ 68
8 todos 0x00000001017e07fa -[RCTModuleMethod invokeWithBridge:module:arguments:]+ 578
9 todos 0x00000001017e2a84 _ZN8facebook5reactL11invokeInnerEP9RCTBridgeP13RCTModuleDatajRKN5folly7dynamicE + 246
10 todos 0x00000001017e280c ___ZN8facebook5react15RCTNativeModule6invokeEjON5folly7dynamicEi_block_invoke + 78
11 libdispatch.dylib 0x00000001025b5f11 _dispatch_call_block_and_release + 12
12 libdispatch.dylib 0x00000001025b6e8e _dispatch_client_callout + 8
13 libdispatch.dylib 0x00000001025bd6fd _dispatch_lane_serial_drain + 788
14 libdispatch.dylib 0x00000001025be28f _dispatch_lane_invoke + 422
15 libdispatch.dylib 0x00000001025c9b65 _dispatch_workloop_worker_thread + 719
16 libsystem_pthread.dylib 0x00007fff51c08a3d _pthread_wqthread + 290
17 libsystem_pthread.dylib 0x00007fff51c07b77 start_wqthread + 15
)
libc++abi.dylib:terminating with uncaught exception of type NSException
(lldb)

Tips RN release js console

  • AppDelegate.m #import <React/RCTLog.h>

  • -(BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions RCTSetLogThreshold(RCTLogLevelTrace);

    3

    iOS release RN

    global.ErrorUtils.setGlobalHandler((e) => {
    console.log(e);
    let message = { name:e.name,

              message:e.message,
              stack:e.stack

    };
    axios.get(' http://192.168.1.100:8888/test.php' , {

    params:{ 'message':JSON.stringify(message) }

    }).then(function(response) {

        console.log(response)

    }).catch(function(error) {
    console.log(error)
    });
    }, true)

    iOS crash

    iOS bundle js
    RN release log

RN crash Native RN crash Native RN crash Native

RN crash js webpack crash RN crash RN crash sourceMap

2.7.3.1 js

RN DEBUG js RELEASE

RN ErrorUtils

/**
 * Copyright(c) Facebook, Inc. and its affiliates.
 *
 * This source code is licensed under the MIT license found in the
 * LICENSE file in the root directory of this source tree.
 *
 * @format
 * @flow strict
 * @polyfill
 */

let _inGuard = 0;

type ErrorHandler =(error:mixed, isFatal:boolean) => void;
type Fn<Args, Return> =(...Args) => Return;

/**
 * This is the error handler that is called when we encounter an exception
 * when loading a module. This will report any errors encountered before
 * ExceptionsManager is configured.
 */
let _globalHandler:ErrorHandler = function onError(
  e:mixed,
  isFatal:boolean,

) {
throw e;
};

/**
 * The particular require runtime that we are using looks for a global
 * `ErrorUtils` object and if it exists, then it requires modules with the
 * error handler specified via ErrorUtils.setGlobalHandler by calling the
 * require function with applyWithGuard. Since the require module is loaded
 * before any of the modules, this ErrorUtils must be defined(and the handler
 * set) globally before requiring anything.
 */
const ErrorUtils = {
  setGlobalHandler(fun:ErrorHandler):void {
    _globalHandler = fun;
  },
  getGlobalHandler():ErrorHandler {
    return _globalHandler;
  },
  reportError(error:mixed):void {
    _globalHandler && _globalHandler(error, false);
  },
  reportFatalError(error:mixed):void {
    //NOTE:This has an untyped call site in Metro.
    _globalHandler && _globalHandler(error, true);
  },
  applyWithGuard<TArgs:$ReadOnlyArray<mixed>, TOut>(
    fun:Fn<TArgs, TOut>,
    context?:?mixed,
    args?:?TArgs,
    //Unused, but some code synced from www sets it to null.
    unused_onError?:null,
    //Some callers pass a name here, which we ignore.
    unused_name?:?string,
 ):?TOut {
    try {
      _inGuard++;
      //$FlowFixMe:TODO T48204745(1) apply(context, null) is fine.(2) array -> rest array should work
      return fun.apply(context, args);
    } catch(e) {
      ErrorUtils.reportError(e);
    } finally {
      _inGuard--;
    }
    return null;
  },
  applyWithGuardIfNeeded<TArgs:$ReadOnlyArray<mixed>, TOut>(
    fun:Fn<TArgs, TOut>,
    context?:?mixed,
    args?:?TArgs,
 ):?TOut {
    if(ErrorUtils.inGuard()) {
      //$FlowFixMe:TODO T48204745(1) apply(context, null) is fine.(2) array -> rest array should work
      return fun.apply(context, args);
    } else {
      ErrorUtils.applyWithGuard(fun, context, args);
    }
    return null;
  },
  inGuard():boolean {
    return !!_inGuard;
  },
  guard<TArgs:$ReadOnlyArray<mixed>, TOut>(
    fun:Fn<TArgs, TOut>,
    name?:?string,
    context?:?mixed,
 ):?(...TArgs) => ?TOut {
    //TODO:(moti) T48204753 Make sure this warning is never hit and remove it - types
    //should be sufficient.
    if(typeof fun !== 'function') {
      console.warn('A function must be passed to ErrorUtils.guard, got ', fun);
      return null;
    }
    const guardName = name ?? fun.name ?? '<generated guard>';
    function guarded(...args:TArgs):?TOut {
      return ErrorUtils.applyWithGuard(
        fun,
        context ?? this,
        args,
        null,
        guardName,
     );
    }

    return guarded;
  },
};

global.ErrorUtils = ErrorUtils;

export type ErrorUtilsT = typeof ErrorUtils;

RN global.ErrorUtils

global.ErrorUtils.setGlobalHandler(e => {
   //e.name e.message e.stack
}, true);
2.7.3.2
 RN   crash              **React Error Boundaries** [  ](https://zh-hans.reactjs.org/docs/error-boundaries.html)
    JavaScript       React                    [](https://github.com/facebook/react/issues/4026) [     ](https://github.com/facebook/react/issues/6895) [](https://github.com/facebook/react/issues/8579)                    React              React                                  

UI JavaScript React 16

    React        **                    JavaScript               UI**                                                  
                     constructor   render   
  • Event handlers

  • Asynchronous code setTimeout promise

  • Server side rendering

  • Errors thrown in the error boundary itself(rather than its children)

                                   App crash                             

    RN crash 2 js js

2.7.4 RN Crash

SourceMap SourceMap

SourceMap mozilla source-map RN crash

 NodeJS        

var fs = require('fs');
var sourceMap = require('source-map');
var arguments = process.argv.splice(2);

function parseJSError(aLine, aColumn) {
    fs.readFile('./index.ios.map', 'utf8', function(err, data) {
        const whatever =  sourceMap.SourceMapConsumer.with(data, null, consumer => {
            //  crash         
            let parseData = consumer.originalPositionFor({
                line:parseInt(aLine),
                column:parseInt(aColumn)
            });
            //     
            console.log(parseData);
            //     
            fs.writeFileSync('./parsed.txt', JSON.stringify(parseData) + '\n', 'utf8', function(err) {  
                if(err) {  
                    console.log(err);
                }
            });
        });
    });
}

var line = arguments[0];
var column = arguments[1];
parseJSError(line, column);

          todos    
  1. Text crash

    <Text style={styles.sectionTitle} onPress={()=>{1+qw;}}>Debug

  2. RN bundle sourceMap ,

    react-native bundle --entry-file index.js --platform android --dev false --bundle-output release_ios/main.jsbundle --assets-dest release_iOS --sourcemap-output release_ios/index.android.map;

         iterm2    alias         `.zshrc`   

    alias RNRelease='react-native bundle --entry-file index.js --platform ios --dev false --bundle-output release_ios/main.jsbundle --assets-dest release_iOS --sourcemap-output release_ios/index.ios.map;' # RN Release

  3. js bundle Xcode

  4. crash Node

    node index.js 397 1822

  5. 2020-06-23-RNCrashLogAnalysis.png

2.7.5 SourceMap
       RN      crash                                     RN stack trace           
    • source map
    • install
  1. RN RN crash RN stack trace Native stack trace

  2. souece map RN

  3. SourceMap iOS Android SoureceMap os

3. KSCrash

     Crash                
  • KSCrashInstallation NSURLProtocol sink

    carbon(17).png

    carbon(18).png

  • sink CMCrashReporterSink KSCrashReportFilter defaultCrashReportFilterSetAppleFmt

    carbon(19).png

    defaultCrashReportFilterSetAppleFmt KSCrashReportFilterPipeline filterWithFilters

    CMCrashReportFilterAppleFmt KSCrashReportFilterAppleFmt KSCrashReportFilter Crash

    carbon(20).png

    carbon(21).png
    carbon(22).png

  • APM Crash KSCrash Crash SESSION_ID App App App

    carbon(23).png
    carbon(24).png

    installKSCrash [[CMCrashInstallation sharedInstance]sendAllReportsWithCompletion:nil]

    carbon(25).png

      `KSCrashInstallation`   `sink`     `KSCrash`             `KSCrash`   `sendAllReportsWithCompletion`        

    carbon(26).png

             `sendReports:onCompletion:`     

    carbon(27).png

      `[self.sink filterReports:onCompletion:]`        `CMCrashInstallation`      `sink` getter          `CMCrashReporterSink`     `defaultCrashReportFilterSetAppleFmt`              

    carbon(28).png

                **filters**        **self**     `CMCrashReporterSink`          `[self.sink filterReports:onCompletion:]`        `CMCrashReporterSink`                 `kscrash_callCompletion(onCompletion, reports, YES, nil);`    `KSCrash`       Crash                  

    carbon(29).png

       KSCrash           crash         crash                         c       json      App            crash       crash              key value          APM           crash         

4.

crash

4.1 .dSYM

.dSYM debugging symbol symbols Xcode .dSYM debug .dSYM Build Settings -> Build Options -> Debug Information Format DWARF DWARF with dSYM File .dSYM

 App                 `.dSYM`    

.dSYM DWARF Test.app.dSYM/Contents/Resources/DWARF/Test DWARF

.dSYM Mach-O .dSYM

2020-05-29-DYSMStructure.png

4.2 DWARF

DWARF is a debugging file format used by many compilers and debuggers to support source level debugging. It addresses the requirements of a number of procedural languages, such as C, C++, and Fortran, and is designed to be extensible to other languages. DWARF is architecture independent and applicable to any processor or operating system. It is widely used on Unix, Linux and other operating systems, as well as in stand-alone environments.

*DWARF * C C++ Fortran DWARF Unix Linux

DWARF Debugging With Arbitrary Record Formats

DWARF

                                       c                                DWARF              DWARF               DIE Debugging Information Entry     DIE            DIE                                html xml        DIE             DIE           DIE     DIE                                            DIE                  

DWARF

.debug_loc DW_AT_location
.debug_macinfo
.debug_pubnames
.debug_pubtypes
.debug_ranges DW_AT_ranges
.debug_str .debug_info
.debug_types
DW_TAG_class_type
DW_TAG_structure_type
DW_TAG_union_type
DW_TAG_enumeration_type
DW_TAG_typedef typedef
DW_TAG_array_type
DW_TAG_subrange_type
DW_TAG_inheritance
DW_TAG_member
DW_TAG_subprogram
DW_TAG_formal_parameter
DW_TAG_name
DW_TAG_type
DW_TAG_artifical
DW_TAG_sibling
DW_TAG_data_memver_location
DW_TAG_virtuality
  DWARF            `.dSYM`       DWARF          

dwarfdump -F --debug-info Test.app.dSYM/Contents/Resources/DWARF/Test > debug-info.txt



Test.app.dSYM/Contents/Resources/DWARF/Test:   file format Mach-O arm64

.debug_info contents:
0x00000000:Compile Unit:length = 0x0000004f version = 0x0004 abbr_offset = 0x0000 addr_size = 0x08(next unit at 0x00000053)

0x0000000b:DW_TAG_compile_unit
              DW_AT_producer [DW_FORM_strp]  ("Apple clang version 11.0.3(clang-1103.0.32.62)")
              DW_AT_language [DW_FORM_data2]  (DW_LANG_ObjC)
              DW_AT_name [DW_FORM_strp]  ("_Builtin_stddef_max_align_t")
              DW_AT_stmt_list [DW_FORM_sec_offset]  (0x00000000)
              DW_AT_comp_dir [DW_FORM_strp]  ("/Users/lbp/Desktop/Test")
              DW_AT_APPLE_major_runtime_vers [DW_FORM_data1]  (0x02)
              DW_AT_GNU_dwo_id [DW_FORM_data8]  (0x392b5344d415340c)

0x00000027:  DW_TAG_module
                DW_AT_name [DW_FORM_strp]  ("_Builtin_stddef_max_align_t")
                DW_AT_LLVM_config_macros [DW_FORM_strp]  ("\"-DDEBUG=1\" \"-DOBJC_OLD_DISPATCH_PROTOTYPES=1\"")
                DW_AT_LLVM_include_path [DW_FORM_strp]  ("/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/11.0.3/include")
                DW_AT_LLVM_isysroot [DW_FORM_strp]  ("/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk")

0x00000038:    DW_TAG_typedef
                  DW_AT_type [DW_FORM_ref4]  (0x0000004b "long double")
                  DW_AT_name [DW_FORM_strp]  ("max_align_t")
                  DW_AT_decl_file [DW_FORM_data1]  ("/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/11.0.3/include/__stddef_max_align_t.h")
                  DW_AT_decl_line [DW_FORM_data1]  (16)

0x00000043:    DW_TAG_imported_declaration
                  DW_AT_decl_file [DW_FORM_data1]  ("/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/11.0.3/include/__stddef_max_align_t.h")
                  DW_AT_decl_line [DW_FORM_data1]  (27)
                  DW_AT_import [DW_FORM_ref_addr]  (0x0000000000000027)

0x0000004a:    NULL

0x0000004b:  DW_TAG_base_type
                DW_AT_name [DW_FORM_strp]  ("long double")
                DW_AT_encoding [DW_FORM_data1]  (DW_ATE_float)
                DW_AT_byte_size [DW_FORM_data1]  (0x08)

0x00000052:  NULL
0x00000053:Compile Unit:length = 0x000183dc version = 0x0004 abbr_offset = 0x0000 addr_size = 0x08(next unit at 0x00018433)

0x0000005e:DW_TAG_compile_unit
              DW_AT_producer [DW_FORM_strp]  ("Apple clang version 11.0.3(clang-1103.0.32.62)")
              DW_AT_language [DW_FORM_data2]  (DW_LANG_ObjC)
              DW_AT_name [DW_FORM_strp]  ("Darwin")
              DW_AT_stmt_list [DW_FORM_sec_offset]  (0x000000a7)
              DW_AT_comp_dir [DW_FORM_strp]  ("/Users/lbp/Desktop/Test")
              DW_AT_APPLE_major_runtime_vers [DW_FORM_data1]  (0x02)
              DW_AT_GNU_dwo_id [DW_FORM_data8]  (0xa4a1d339379e18a5)

0x0000007a:  DW_TAG_module
                DW_AT_name [DW_FORM_strp]  ("Darwin")
                DW_AT_LLVM_config_macros [DW_FORM_strp]  ("\"-DDEBUG=1\" \"-DOBJC_OLD_DISPATCH_PROTOTYPES=1\"")
                DW_AT_LLVM_include_path [DW_FORM_strp]  ("/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk/usr/include")
                DW_AT_LLVM_isysroot [DW_FORM_strp]  ("/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk")

0x0000008b:    DW_TAG_module
                  DW_AT_name [DW_FORM_strp]  ("C")
                  DW_AT_LLVM_config_macros [DW_FORM_strp]  ("\"-DDEBUG=1\" \"-DOBJC_OLD_DISPATCH_PROTOTYPES=1\"")
                  DW_AT_LLVM_include_path [DW_FORM_strp]  ("/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk/usr/include")
                  DW_AT_LLVM_isysroot [DW_FORM_strp]  ("/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk")

0x0000009c:      DW_TAG_module
                    DW_AT_name [DW_FORM_strp]  ("fenv")
                    DW_AT_LLVM_config_macros [DW_FORM_strp]  ("\"-DDEBUG=1\" \"-DOBJC_OLD_DISPATCH_PROTOTYPES=1\"")
                    DW_AT_LLVM_include_path [DW_FORM_strp]  ("/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk/usr/include")
                    DW_AT_LLVM_isysroot [DW_FORM_strp]  ("/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk")

0x000000ad:        DW_TAG_enumeration_type
                      DW_AT_type [DW_FORM_ref4]  (0x00017276 "unsigned int")
                      DW_AT_byte_size [DW_FORM_data1]  (0x04)
                      DW_AT_decl_file [DW_FORM_data1]  ("/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk/usr/include/fenv.h")
                      DW_AT_decl_line [DW_FORM_data1]  (154)

0x000000b5:          DW_TAG_enumerator
                        DW_AT_name [DW_FORM_strp]  ("__fpcr_trap_invalid")
                        DW_AT_const_value [DW_FORM_udata]  (256)

0x000000bc:          DW_TAG_enumerator
                        DW_AT_name [DW_FORM_strp]  ("__fpcr_trap_divbyzero")
                        DW_AT_const_value [DW_FORM_udata]  (512)

0x000000c3:          DW_TAG_enumerator
                        DW_AT_name [DW_FORM_strp]  ("__fpcr_trap_overflow")
                        DW_AT_const_value [DW_FORM_udata]  (1024)

0x000000ca:          DW_TAG_enumerator
                        DW_AT_name [DW_FORM_strp]  ("__fpcr_trap_underflow")
//......
0x000466ee:  DW_TAG_subprogram
                DW_AT_name [DW_FORM_strp]  ("CFBridgingRetain")
                DW_AT_decl_file [DW_FORM_data1]  ("/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSObject.h")
                DW_AT_decl_line [DW_FORM_data1]  (105)
                DW_AT_prototyped [DW_FORM_flag_present]  (true)
                DW_AT_type [DW_FORM_ref_addr]  (0x0000000000019155 "CFTypeRef")
                DW_AT_inline [DW_FORM_data1]  (DW_INL_inlined)

0x000466fa:    DW_TAG_formal_parameter
                  DW_AT_name [DW_FORM_strp]  ("X")
                  DW_AT_decl_file [DW_FORM_data1]  ("/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSObject.h")
                  DW_AT_decl_line [DW_FORM_data1]  (105)
                  DW_AT_type [DW_FORM_ref4]  (0x00046706 "id")

0x00046705:    NULL

0x00046706:  DW_TAG_typedef
                DW_AT_type [DW_FORM_ref4]  (0x00046711 "objc_object*")
                DW_AT_name [DW_FORM_strp]  ("id")
                DW_AT_decl_file [DW_FORM_data1]  ("/Users/lbp/Desktop/Test/Test/NetworkAPM/NSURLResponse+cm_FetchStatusLineFromCFNetwork.m")
                DW_AT_decl_line [DW_FORM_data1]  (44)

0x00046711:  DW_TAG_pointer_type
                DW_AT_type [DW_FORM_ref4]  (0x00046716 "objc_object")

0x00046716:  DW_TAG_structure_type
                DW_AT_name [DW_FORM_strp]  ("objc_object")
                DW_AT_byte_size [DW_FORM_data1]  (0x00)

0x0004671c:    DW_TAG_member
                  DW_AT_name [DW_FORM_strp]  ("isa")
                  DW_AT_type [DW_FORM_ref4]  (0x00046727 "objc_class*")
                  DW_AT_data_member_location [DW_FORM_data1]  (0x00)
//......

                  DIE                                                           DIE                

debug_line

dwarfdump -F --debug-line Test.app.dSYM/Contents/Resources/DWARF/Test > debug-inline.txt



Test.app.dSYM/Contents/Resources/DWARF/Test:   file format Mach-O arm64

.debug_line contents:
debug_line[0x00000000]
Line table prologue:
    total_length:0x000000a3
         version:4
 prologue_length:0x0000009a
 min_inst_length:1
max_ops_per_inst:1
 default_is_stmt:1
       line_base:-5
      line_range:14
     opcode_base:13
standard_opcode_lengths[DW_LNS_copy]= 0
standard_opcode_lengths[DW_LNS_advance_pc]= 1
standard_opcode_lengths[DW_LNS_advance_line]= 1
standard_opcode_lengths[DW_LNS_set_file]= 1
standard_opcode_lengths[DW_LNS_set_column]= 1
standard_opcode_lengths[DW_LNS_negate_stmt]= 0
standard_opcode_lengths[DW_LNS_set_basic_block]= 0
standard_opcode_lengths[DW_LNS_const_add_pc]= 0
standard_opcode_lengths[DW_LNS_fixed_advance_pc]= 1
standard_opcode_lengths[DW_LNS_set_prologue_end]= 0
standard_opcode_lengths[DW_LNS_set_epilogue_begin]= 0
standard_opcode_lengths[DW_LNS_set_isa]= 1
include_directories[ 1]= "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/11.0.3/include"
file_names[ 1]:
           name:"__stddef_max_align_t.h"
      dir_index:1
       mod_time:0x00000000
         length:0x00000000

Address            Line   Column File   ISA Discriminator Flags
------------------ ------ ------ ------ --- ------------- -------------
0x0000000000000000      1      0      1   0             0  is_stmt end_sequence
debug_line[0x000000a7]
Line table prologue:
    total_length:0x0000230a
         version:4
 prologue_length:0x00002301
 min_inst_length:1
max_ops_per_inst:1
 default_is_stmt:1
       line_base:-5
      line_range:14
     opcode_base:13
standard_opcode_lengths[DW_LNS_copy]= 0
standard_opcode_lengths[DW_LNS_advance_pc]= 1
standard_opcode_lengths[DW_LNS_advance_line]= 1
standard_opcode_lengths[DW_LNS_set_file]= 1
standard_opcode_lengths[DW_LNS_set_column]= 1
standard_opcode_lengths[DW_LNS_negate_stmt]= 0
standard_opcode_lengths[DW_LNS_set_basic_block]= 0
standard_opcode_lengths[DW_LNS_const_add_pc]= 0
standard_opcode_lengths[DW_LNS_fixed_advance_pc]= 1
standard_opcode_lengths[DW_LNS_set_prologue_end]= 0
standard_opcode_lengths[DW_LNS_set_epilogue_begin]= 0
standard_opcode_lengths[DW_LNS_set_isa]= 1
include_directories[ 1]= "/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk/usr/include"
include_directories[ 2]= "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/11.0.3/include"
include_directories[ 3]= "/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk/usr/include/sys"
include_directories[ 4]= "/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk/usr/include/mach"
include_directories[ 5]= "/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk/usr/include/libkern"
include_directories[ 6]= "/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk/usr/include/architecture"
include_directories[ 7]= "/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk/usr/include/sys/_types"
include_directories[ 8]= "/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk/usr/include/_types"
include_directories[ 9]= "/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk/usr/include/arm"
include_directories[10]= "/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk/usr/include/sys/_pthread"
include_directories[11]= "/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk/usr/include/mach/arm"
include_directories[12]= "/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk/usr/include/libkern/arm"
include_directories[13]= "/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk/usr/include/uuid"
include_directories[14]= "/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk/usr/include/netinet"
include_directories[15]= "/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk/usr/include/netinet6"
include_directories[16]= "/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk/usr/include/net"
include_directories[17]= "/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk/usr/include/pthread"
include_directories[18]= "/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk/usr/include/mach_debug"
include_directories[19]= "/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk/usr/include/os"
include_directories[20]= "/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk/usr/include/malloc"
include_directories[21]= "/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk/usr/include/bsm"
include_directories[22]= "/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk/usr/include/machine"
include_directories[23]= "/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk/usr/include/mach/machine"
include_directories[24]= "/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk/usr/include/secure"
include_directories[25]= "/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk/usr/include/xlocale"
include_directories[26]= "/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk/usr/include/arpa"
file_names[ 1]:
           name:"fenv.h"
      dir_index:1
       mod_time:0x00000000
         length:0x00000000
file_names[ 2]:
           name:"stdatomic.h"
      dir_index:2
       mod_time:0x00000000
         length:0x00000000
file_names[ 3]:
           name:"wait.h"
      dir_index:3
       mod_time:0x00000000
         length:0x00000000
//......
Address            Line   Column File   ISA Discriminator Flags
------------------ ------ ------ ------ --- ------------- -------------
0x000000010000b588     14      0      2   0             0  is_stmt
0x000000010000b5b4     16      5      2   0             0  is_stmt prologue_end
0x000000010000b5d0     17     11      2   0             0  is_stmt
0x000000010000b5d4      0      0      2   0             0 
0x000000010000b5d8     17      5      2   0             0 
0x000000010000b5dc     17     11      2   0             0 
0x000000010000b5e8     18      1      2   0             0  is_stmt
0x000000010000b608     20      0      2   0             0  is_stmt
0x000000010000b61c     22      5      2   0             0  is_stmt prologue_end
0x000000010000b628     23      5      2   0             0  is_stmt
0x000000010000b644     24      1      2   0             0  is_stmt
0x000000010000b650     15      0      1   0             0  is_stmt
0x000000010000b65c     15     41      1   0             0  is_stmt prologue_end
0x000000010000b66c     11      0      2   0             0  is_stmt
0x000000010000b680     11     17      2   0             0  is_stmt prologue_end
0x000000010000b6a4     11     17      2   0             0  is_stmt end_sequence
debug_line[0x0000def9]
Line table prologue:
    total_length:0x0000015a
         version:4
 prologue_length:0x000000eb
 min_inst_length:1
max_ops_per_inst:1
 default_is_stmt:1
       line_base:-5
      line_range:14
     opcode_base:13
standard_opcode_lengths[DW_LNS_copy]= 0
standard_opcode_lengths[DW_LNS_advance_pc]= 1
standard_opcode_lengths[DW_LNS_advance_line]= 1
standard_opcode_lengths[DW_LNS_set_file]= 1
standard_opcode_lengths[DW_LNS_set_column]= 1
standard_opcode_lengths[DW_LNS_negate_stmt]= 0
standard_opcode_lengths[DW_LNS_set_basic_block]= 0
standard_opcode_lengths[DW_LNS_const_add_pc]= 0
standard_opcode_lengths[DW_LNS_fixed_advance_pc]= 1
standard_opcode_lengths[DW_LNS_set_prologue_end]= 0
standard_opcode_lengths[DW_LNS_set_epilogue_begin]= 0
standard_opcode_lengths[DW_LNS_set_isa]= 1
include_directories[ 1]= "Test"
include_directories[ 2]= "Test/NetworkAPM"
include_directories[ 3]= "/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.5.sdk/usr/include/objc"
file_names[ 1]:
           name:"AppDelegate.h"
      dir_index:1
       mod_time:0x00000000
         length:0x00000000
file_names[ 2]:
           name:"JMWebResourceURLProtocol.h"
      dir_index:2
       mod_time:0x00000000
         length:0x00000000
file_names[ 3]:
           name:"AppDelegate.m"
      dir_index:1
       mod_time:0x00000000
         length:0x00000000
file_names[ 4]:
           name:"objc.h"
      dir_index:3
       mod_time:0x00000000
         length:0x00000000
//......

 debug\_line                      AppDelegate     

4.3 symbols

               Symbol               Symbol Name                                        

** Symbol Table** Symbol Value

<    > <    > <  > [<      >]

4.4 ** **

image frame

Binary Images

   crash             Binary Images   

//...
Binary Images:
0x102fe0000 - 0x102ff3fff Test arm64  <37eaa57df2523d95969e47a9a1d69ce5> /var/containers/Bundle/Application/643F0DFE-A710-4136-A278-A89D780B7208/Test.app/Test
0x1030e0000 - 0x1030ebfff libobjc-trampolines.dylib arm64  <181f3aa866d93165ac54344385ac6e1d> /usr/lib/libobjc-trampolines.dylib
0x103204000 - 0x103267fff dyld arm64  <6f1c86b640a3352a8529bca213946dd5> /usr/lib/dyld
0x189a78000 - 0x189a8efff libsystem_trace.dylib arm64  <b7477df8f6ab3b2b9275ad23c6cc0b75> /usr/lib/system/libsystem_trace.dylib
//...

 Crash     Binary Images      Image              image    arm    uuid image    

crash

Last Exception Backtrace:
//...
5   Test                              0x102fe592c -[ViewController testMonitorCrash]+ 22828(ViewController.mm:58)

Binary Images:
0x102fe0000 - 0x102ff3fff Test arm64  <37eaa57df2523d95969e47a9a1d69ce5> /var/containers/Bundle/Application/643F0DFE-A710-4136-A278-A89D780B7208/Test.app/Test

frame 5 0x102fe592c - 0x102fe0000

atos 0x102fe0000 image 0x102fe592c frame

atos -o Test.app.dSYM/Contents/Resources/DWARF/Test-arch arm64 -l 0x102fe0000 0x102fe592c

4.5 UUID

  • crash UUID

    grep --after-context=2 "Binary Images:" *.crash
    
    Test  5-28-20, 7-47 PM.crash:Binary Images:
    Test  5-28-20, 7-47 PM.crash-0x102fe0000 - 0x102ff3fff Test arm64  <37eaa57df2523d95969e47a9a1d69ce5> /var/containers/Bundle/Application/643F0DFE-A710-4136-A278-A89D780B7208/Test.app/Test
    Test  5-28-20, 7-47 PM.crash-0x1030e0000 - 0x1030ebfff libobjc-trampolines.dylib arm64  <181f3aa866d93165ac54344385ac6e1d> /usr/lib/libobjc-trampolines.dylib
    -
    Test.crash:Binary Images:
    Test.crash-0x102fe0000 - 0x102ff3fff Test arm64  <37eaa57df2523d95969e47a9a1d69ce5> /var/containers/Bundle/Application/643F0DFE-A710-4136-A278-A89D780B7208/Test.app/Test
    Test.crash-0x1030e0000 - 0x1030ebfff libobjc-trampolines.dylib arm64  <181f3aa866d93165ac54344385ac6e1d> /usr/lib/libobjc-trampolines.dylib

    Test App UUID 37eaa57df2523d95969e47a9a1d69ce5.

  • .dSYM UUID

    dwarfdump --uuid Test.app.dSYM
    UUID:37EAA57D-F252-3D95-969E-47A9A1D69CE5(arm64) Test.app.dSYM/Contents/Resources/DWARF/Test
  • app UUID

    dwarfdump --uuid Test.app/Test
    UUID:37EAA57D-F252-3D95-969E-47A9A1D69CE5(arm64) Test.app/Test

4.6 Crash

             crash App                   crash                                                    

  [.dSYM  ](#dSYM)     **        dSYM                         **    .dSYM       crash log     bundle id version      

Crash Xcode -> Window -> Devices and Simulators Crash App

app .dSYM ~/Library/Developer/Xcode/Archives

   2  
  • symbolicatecrash

    symbolicatecrash Xcode crash

    find /Applications/Xcode.app -name symbolicatecrash -type f

        `iPhoneSimulator.platform`      

    /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/Library/PrivateFrameworks/DVTFoundation.framework/symbolicatecrash

    symbolicatecrash app dSYM crash

    ./symbolicatecrash Test.crash Test.dSYM > Test.crash

             `Error:"DEVELOPER_DIR" is not defined at ./symbolicatecrash line 69.`               

    export DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer
  • atos

    symbolicatecrash atos .crash .dSYM .crash .app

    -l

    xcrun atos -o Test.app.dSYM/Contents/Resources/DWARF/Test -arch armv7 -l 0x1023c592c

    .app .dSYM xxx xx

    atos -arch architecture -o binary -l xxx xx

  App          App                   APM                 crash     `.dSYM`                        **UUID**    

4.7

     Xcode                                          `/Users/      /Library/Developer/Xcode/iOS DeviceSupport`                               

/Users/      /Library/Developer/Xcode/iOS DeviceSupport/

2020-05-28-SymbolicateLib.png

5.

5.1 ELK
              ELK    ELK   Elasticsearch Logstash Kibana          Elasticsearch           Restful                   Logstash                     /  /MQ                             /MQ/Redis/ElasticsSearch/Kafka  Kibana     Elasticserarch                             ELK                     

                                                                                                                                                                                                                    Logstash                                   Kafka   Redis        Logstash   Kafka   Redis          ES            Kibana                                       

ELK

    ELK              
  • Logstash ES Kafka Logstash Kafka Kafka

  • Logstash Kafka ES

  • n Logstash n Kafka n Logstash m App Tomcat Nginx

    Elasticsearch Elastic APM

2020-06-17-ElastiicsearchAPM.png

5.2

Crash log Kibana crash

2020-06-14-CrashLogSymbolicate.png

         APM SDK    crash log -> Kafka    -> Mac            ->      Kafka ->                          

            App           App            crash             .dSYM        App                   

2                      Xcode     runScript          release      dSYM  

           wax-cli        iOS SDK iOS App Android SDK Android App Node React React Native                       Unit Test Lint                                                                       .dSYM                 AppName + Version   key value   .dSYM     

                              crash                      

2020-06-17-APMServerArch.png

  • Symbolication Service Prism crash report
  • mass crash report dsym index dsym crash report hash hash mass
  • Prism crash report dsym index dsym crash report crash report Prism
  • Mass (/)
  • candle wax-cli candle

2020-06-17-symolication_flow.png

                       NodeJS     iOS             Mac mini                   worker                   crash log                                 mac mini            worker          

2020-06-17-APMServerWorker.png
master slave master .dSYM crash cache mass 2 symbolocate worker .dSYM

2020-06-17-SymbolicateServerArch.png

APM

  1. APM

  2. crash ANR hot fix

  3. SDK

  4. ANR

    carbon(30).png

  5. APM
    2020-06-17-APMStructure.jpg

*      SDK    sessionId        
*   wax                       wax          
  1. APM Hermes Flink SQL InfluxDB

References